-- Logs begin at Fri 2022-05-27 15:22:47 UTC, end at Fri 2022-05-27 16:52:58 UTC. -- May 27 16:05:36.234757 dv-cinder-840493-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 16:05:36.244297 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 16:05:36 2022] *** May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: nodename: dv-cinder-840493-2-iscsi May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: machine: x86_64 May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: clock source: unix May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: pcre jit disabled May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: detected number of CPU cores: 4 May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: current working directory: / May 27 16:05:36.246329 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: detected binary path: /usr/bin/uwsgi-core May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: your processes number limit is 47680 May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: your memory page size is 4096 bytes May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: detected max file descriptor number: 2048 May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: lock engine: pthread robust mutexes May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: thunder lock: enabled May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 16:05:36.246866 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 16:05:36.290908 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: Python main interpreter initialized at 0x559b704309c0 May 27 16:05:36.290908 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: python threads support enabled May 27 16:05:36.290908 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: your server socket listen backlog is limited to 100 connections May 27 16:05:36.290908 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: your mercy for graceful operations on workers is 90 seconds May 27 16:05:36.291258 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: mapped 403077 bytes (393 KB) for 2 cores May 27 16:05:36.291327 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: *** Operational MODE: preforking *** May 27 16:05:36.291589 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: *** uWSGI is running in multiple interpreter mode *** May 27 16:05:36.291589 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: spawned uWSGI master process (pid: 80554) May 27 16:05:36.291788 dv-cinder-840493-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 16:05:36.293154 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: spawned uWSGI worker 1 (pid: 80565, cores: 1) May 27 16:05:36.293154 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: spawned uWSGI worker 2 (pid: 80566, cores: 1) May 27 16:05:36.293154 dv-cinder-840493-2-iscsi devstack@keystone.service[80554]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 16:05:41.946627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 16:05:41.946627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: from cryptography.utils import int_from_bytes May 27 16:05:41.947308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 16:05:41.947308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: from cryptography.utils import int_from_bytes May 27 16:05:41.956300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 16:05:41.956300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: from cryptography.utils import int_from_bytes May 27 16:05:41.956765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 16:05:41.956765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: from cryptography.utils import int_from_bytes May 27 16:05:41.996248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 16:05:41.996442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 16:05:41.996635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] command line args: [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 16:05:41.996902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 16:05:41.997081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ================================================================================ {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 16:05:41.997331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] admin_token = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.997600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] config_dir = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.997789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.997971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] config_source = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.998152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] debug = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.998351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:41.999710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_config_append = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.000125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.000125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_dir = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.000263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_file = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.000455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.000703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.001004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.001232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.001434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.001748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.002002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.002245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.002478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.002711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.002945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.003173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.003418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.003662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.003822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 16:05:42.003945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.004023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 16:05:42.004187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] command line args: [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 16:05:42.004426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.004517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 16:05:42.004608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] publish_errors = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.004838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ================================================================================ {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 16:05:42.005024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] admin_token = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] config_dir = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] config_source = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] debug = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.005936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] standard_threads = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] use_journal = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] use_json = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.006961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] use_stderr = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_config_append = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] use_syslog = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_dir = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_file = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.007865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.007952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.008063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.008177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.008316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.008399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.008563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.008655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.008744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.008839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.008982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.009058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.009156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.009253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.009433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.009525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.009669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.009744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.009902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.009999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.010138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.010220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.010372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.010450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.010620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.010694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.010866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.010956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.011151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.011226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.011386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.011636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.011719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.011818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] publish_errors = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.011893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.012049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.012130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.012280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.012367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.012458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.012572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.012702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.012833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.012937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.013087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.013163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] standard_threads = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.013349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.013436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.013587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.013670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.013812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.013886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.013990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] use_journal = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.014064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.014167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] use_json = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.014317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.014394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] use_stderr = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.014496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.014578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] use_syslog = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.014727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 16:05:42.014800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.014958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.015915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.016003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.016153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.016234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.016382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.016986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.017976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.018930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.019016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.019160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.019241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.020906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.021948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.022908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.023982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.024923 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.025947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.026881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027776 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.027994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.028943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.029904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.030989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.031961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.032050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.032769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.032850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.external = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.033953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.password = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] auth.token = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.034970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.035940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.036887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.037853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.038951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.039932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.external = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.040947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.password = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] auth.token = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.041928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.042846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.043976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.044921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.045962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.046988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.047972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.048886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.049878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.050858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.051960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.052972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.053949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.054893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.055971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.056992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.057955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.058894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.059988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.060928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061162 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.061926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.062972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.063865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.064864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.065844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.066975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] role.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] role.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.067939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.068917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.069984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.070942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.071995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.072975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.073951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] role.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.074979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] role.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.075904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.076862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.077847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.078879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.079480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.079627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.079764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.079956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.connection = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.080986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.081969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.082917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.083851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.084948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80566) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 16:05:42.085462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.085923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.086172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.086410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.086649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.086885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.087132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.087380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.087612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.087867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.088171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.088435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.connection = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.088682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.088936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.089177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.089441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.089681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.089918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.090153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.090390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.090634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.090869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.091113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.091350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.091598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.091834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.092090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.092324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.092568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.092807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 16:05:42.092978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 16:05:42.116462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.116688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.116908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.118439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.118677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.118904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.125357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.128792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.154355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.154581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.155384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.155619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.207750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.208130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.258298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.258521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.259286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.259521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.324596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.324821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.324912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.325991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.326075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.326200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.350928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.351058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.351137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.351273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.351353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.362875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.362969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.375278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.375519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.375634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.375743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.376890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.377023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.377103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80566) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.377234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.377474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 16:05:42.394706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.394815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.394920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 16:05:42.395000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 16:05:42.395251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.395331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.395449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 16:05:42.395528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 16:05:42.395715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.395808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.395974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.396053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.396225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.396303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.396477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.396563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.407916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.408290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.408552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.408824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.409782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.410063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.410329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.410608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.416823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.418908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.420347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.420626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.420905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.421185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.421492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.421752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.422126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.422205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.422398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.422477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.422728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.422817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.423138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.423292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.423411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.423728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.423999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.424276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.424557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.424834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.425115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.425407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.450093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.451797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.452071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.452150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.453771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.454116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.457562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.457882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.458118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.459331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.459609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.459892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.465274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.465573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.467165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.467448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.470719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.471114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.471482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.472083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.472304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.472492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.472914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.473456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.473646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.473939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.474223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.474506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.474780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.475052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.475321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.475589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.491643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.491919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.494342 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.494622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.496556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.499157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.500156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.500447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.503339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.503628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.506170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.508689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.509745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.512279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.513465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.513764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.514054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.514344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.514627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.514905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.516037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.516337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.516743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.517035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.517333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.517642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.534529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.534808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.535074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.535336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.537084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.537391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.537752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.538018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.543233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.545177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.545476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.546170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.547956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.548365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.548660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.548946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.549227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.549527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.549804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.551350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.551632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.551911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.552188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.552468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.563032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.563870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.564446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.564728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.565003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.565273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.565575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.566351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.566628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.567194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.567475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.567761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.568045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.583604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.585933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.586841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.589145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.590138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.591725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.592448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.593345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.593644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.593923 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.594088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.596481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.596759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.597048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.601967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.603550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.604472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.606114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.606800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.607095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.607373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.607644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.609361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.609655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.609942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.610317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.616704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.616989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.617389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.619497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.619782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.620061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.625130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80566) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.625431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.625709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.626784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.626784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.627037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.627321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.627582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.627852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.628124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.628396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.628484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 16:05:42.628653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.628761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.628920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.629046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.629182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80566) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.629320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.629613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.629885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.630157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.630441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.630711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.630996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.631265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.631532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.631797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.632062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 16:05:42.667792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559b704309c0 pid: 80566 (default app) May 27 16:05:42.671273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1fc1-fa5d-4fe9-9054-ec1b31c51ffe None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:42.671552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559b704309c0 pid: 80565 (default app) May 27 16:05:42.671633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1fc1-fa5d-4fe9-9054-ec1b31c51ffe None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:42.671833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1fc1-fa5d-4fe9-9054-ec1b31c51ffe None None] PATH_INFO: `/v3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:42.676352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Fri May 27 16:05:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:05:51.700727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:51.701294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:51.701421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:51.706325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:05:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:05:51.711477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:51.711886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:51.712050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:51.779939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: INFO dbcounter [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] Registered counter for database keystone May 27 16:05:51.801664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG oslo_db.sqlalchemy.engines [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80566) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 16:05:51.807808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writer thread running {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 16:05:51.985007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG passlib.handlers.bcrypt [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80566) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 16:05:51.985374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG passlib.handlers.bcrypt [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80566) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 16:05:52.041013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:05:52.042735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:05:52.174649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:05:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 16:05:52.322103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:52.392139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: INFO dbcounter [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] Registered counter for database keystone May 27 16:05:52.411589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG oslo_db.sqlalchemy.engines [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80565) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 16:05:52.417821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writer thread running {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 16:05:52.539261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b6d2e-6570-4d78-83b7-2a97431c5cbb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:52.543790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:52.544502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:52.544796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:52.545080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:52.553843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:52.605620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 16:05:52.605919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939f0b0-d206-4c88-be80-b6f0af526213 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:52.616290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Fri May 27 16:05:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:05:52.622458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:52.638073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568e4d-4972-4ba9-bf3f-b5119296d1a4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:52.643014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:52.644132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:52.644421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:52.644700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] PATH_INFO: `/v3/projects/b89eb764079a4814b557772bebc9a51d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:52.646226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] RBAC: Authorizing `identity:get_project(project_id=b89eb764079a4814b557772bebc9a51d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:52.805518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 16:05:52.806161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c50c55-d91c-4c43-97dd-9143fd8d96e3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:52.810519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Fri May 27 16:05:52 2022] GET /identity/v3/projects/b89eb764079a4814b557772bebc9a51d => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:05:54.966678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2493f13-d107-458e-a9e9-47088873dae1 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:54.967166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2493f13-d107-458e-a9e9-47088873dae1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:54.967311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2493f13-d107-458e-a9e9-47088873dae1 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:54.971419 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:05:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:05:54.976894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:54.977293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:54.977496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:54.996620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:05:54.998195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:05:55.006948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:05:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 16:05:55.028926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2493f13-d107-458e-a9e9-47088873dae1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:55.041462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2493f13-d107-458e-a9e9-47088873dae1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:55.045975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:55.046605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:55.047437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:55.047722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:55.055514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:55.057308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15854ab4-c2d6-4fc8-9448-71a7bb819127 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:55.066648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Fri May 27 16:05:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:05:55.072060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:55.081265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95575-0617-42ce-87a2-ecf3fcc40483 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:55.085649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:55.086502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:55.086788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:55.087066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:55.088458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:55.090335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437d9e3-9cc3-4197-ba52-b879273ec824 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:55.091886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Fri May 27 16:05:55 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:05:57.342649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5f16d-6e48-43a5-b6d3-870415ae423b None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.343153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5f16d-6e48-43a5-b6d3-870415ae423b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.343153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5f16d-6e48-43a5-b6d3-870415ae423b None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.344763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:05:57.348495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ff32-a33b-416e-9c43-bb6d91b2febd None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.348902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ff32-a33b-416e-9c43-bb6d91b2febd None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.349124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3ff32-a33b-416e-9c43-bb6d91b2febd None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.349720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.350010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.350290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.350653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:05:57.355384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.355677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.355964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.372029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:05:57.373213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:05:57.389648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG passlib.handlers.bcrypt [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80565) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 16:05:57.390070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG passlib.handlers.bcrypt [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80565) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 16:05:57.437124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:05:57.439418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:05:57.451818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 7/12] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:05:57.474362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.498606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:57.543499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 6/13] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:05:57.566699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.580484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e581eb-b3fb-4933-b229-5c375c00813d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:57.615223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:57.615881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.616174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.616460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.670965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c7b95-bc5d-4df9-ab3b-73f16c1a2974 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:57.673621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:57.684316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:57.685865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:57.707479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:57.708528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.708817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.709103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.709518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:57.721591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:05:57.732737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:05:57.733880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:05:57.735652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:05:57.740413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.784906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 8/15] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 16:05:57.790334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.839266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a73ee-5f91-4536-8394-c5123856d03a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:57.872790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:57.873409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.873698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.873975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.875065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:05:57.875469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:57.876922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:57.888363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:05:57.889179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf765718-1ec3-4f33-a5d4-3010a7eb65f5 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:57.894600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.912023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:57.922163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:57.923235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:57.927297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:57.927297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:57.927297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:57.934525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:57.936015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:05:57.958378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:05:57.959409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:05:57.960403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:05:57.960403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:05:57.960403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:05:57.960403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:05:57.960403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:05:57.966764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:57.994908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b5d2-a5b2-4668-9ffe-cb3ed79145dc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:58.029503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:58.030377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:58.030662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:58.031669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:58.033384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:58.034906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:58.036751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:05:58.041881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:58.068593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a220-0ec6-4e8e-8c30-6789a14631a4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:58.103027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:58.103742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:58.104029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:58.104312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:58.105031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:05:58.105363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:58.106750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:58.123725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:05:58.140618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:58.148278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d4b95-6dfb-4bfb-a4f8-4cab6ad829dc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:58.187576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:58.188510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:58.188799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:58.189081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:58.190446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:58.191962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd62136-0ae6-4ac2-9067-af90f1f88b4b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:58.193660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:05:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:05:58.199540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:05:58.237639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:05:58.308661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611bd8c-0f9c-4a43-9fe6-05aeac9db149 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:05:58.342664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:05:58.343330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:05:58.344106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:05:58.344391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:05:58.345235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.domain.id=default, user.id=e387d1cbdd3947bcb9659065cbae32c0) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:05:58.345596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:05:58.347068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a5e33-f0e9-4cd6-8727-fa652536f315 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:05:58.354093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Fri May 27 16:05:58 2022] GET /identity/v3/role_assignments?user.id=e387d1cbdd3947bcb9659065cbae32c0&scope.domain.id=default&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:00.779473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50a59-8ca4-4527-b761-f9c39c7deb98 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:00.780494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50a59-8ca4-4527-b761-f9c39c7deb98 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:00.781242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50a59-8ca4-4527-b761-f9c39c7deb98 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:00.783330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:00.793333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479c6d-82dc-484f-a001-059265c1fa4b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:00.793791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479c6d-82dc-484f-a001-059265c1fa4b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:00.793973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48479c6d-82dc-484f-a001-059265c1fa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:00.818607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-48479c6d-82dc-484f-a001-059265c1fa4b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:00.819845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-48479c6d-82dc-484f-a001-059265c1fa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:00.910167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:00.962827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50a59-8ca4-4527-b761-f9c39c7deb98 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.047788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c7f1-e79c-4713-8117-e104f8851d74 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.048070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c7f1-e79c-4713-8117-e104f8851d74 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.048275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c7f1-e79c-4713-8117-e104f8851d74 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.049882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:01.063300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.063593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.063876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.090378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:01.091597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:01.129442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50a59-8ca4-4527-b761-f9c39c7deb98 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:01.144117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:01.200534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:01.201371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.201669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.201968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.244019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 16:06:01.324529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.406768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:01.408583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-100a8d95-5773-42e7-997c-dfe6489f6e5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:06:01.439959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:01.440839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:01.442065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:06:01.442065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:06:01.442065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:06:01.442065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:01.442065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 12/26] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:01.442581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9acc-23c7-493b-a3fc-7740d517e51a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.442883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9acc-23c7-493b-a3fc-7740d517e51a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.443169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9acc-23c7-493b-a3fc-7740d517e51a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.444416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 13/27] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:01.449174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e9acc-23c7-493b-a3fc-7740d517e51a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.521750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:01.528567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da32389-43fd-45bf-8af6-0a33d3679a4a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:01.560526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e9acc-23c7-493b-a3fc-7740d517e51a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:01.567038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:01.567857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.568148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.568431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.580868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-c03150ae-1fb5-46ff-b3b1-a37d7f5ea5e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:01.589804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:01.590619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:01.591372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:01.592800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47ff5-3f0b-4c5d-a81e-c63298d61e0b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.592800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47ff5-3f0b-4c5d-a81e-c63298d61e0b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.592800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca47ff5-3f0b-4c5d-a81e-c63298d61e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.604822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:01.605543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.605833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.606113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.606857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:01.607166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:01.609901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b8ed1-8e2c-404d-a242-1072b7174c2e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:01.615621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-bca47ff5-3f0b-4c5d-a81e-c63298d61e0b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:01.617066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bca47ff5-3f0b-4c5d-a81e-c63298d61e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:01.629720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:01.631126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176e8e-fceb-47b3-9e29-2290aa2b0e3b None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.631425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176e8e-fceb-47b3-9e29-2290aa2b0e3b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.631708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e176e8e-fceb-47b3-9e29-2290aa2b0e3b None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.633168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 15/30] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:01.635332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176e8e-fceb-47b3-9e29-2290aa2b0e3b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.637675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:01.700830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 16/31] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:01.703220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.703861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.704288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.737147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:01.738736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:01.744635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:01.775563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e176e8e-fceb-47b3-9e29-2290aa2b0e3b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:01.818499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:01.820011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:01.820433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:01.820911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:01.821912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:01.822489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:01.824169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:01.829361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 17/32] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:01.830107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.841568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 16/33] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:01.842825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:01.932681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:01.959098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a0682-9816-4ddc-87e8-ab7ca3bba531 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:01.960653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1189-319e-4342-8ca9-18f53b23422f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.000331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.001150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.001459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.001744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.002015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.002490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.002680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.002970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.003252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.003887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.003987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.005428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.009537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:02.241610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 17/34] 10.222.0.42 () {60 vars in 1131 bytes} [Fri May 27 16:06:01 2022] POST /identity/v3/roles => generated 219 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:02.242919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:02.279513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 18/35] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:06:01 2022] POST /identity/v3/projects => generated 301 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:02.280790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:02.454725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ee60-fccd-4a3c-9b70-6def12d7dcd1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.478364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b17b7b-0872-4a77-a773-daf2067e9fb4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.490550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:02.520169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.520169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.520169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.520169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.520577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.520577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9c412-2440-4824-855f-0db597bf26c0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.522954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.523849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.524147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.524428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.526120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.527362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba3bd3-bef8-4cc4-84af-96c2ae2ff57d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.529189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 18/36] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:02 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:02.534887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9aac-b982-41f4-985b-91859a6d1111 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.535373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9aac-b982-41f4-985b-91859a6d1111 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.535804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9aac-b982-41f4-985b-91859a6d1111 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.537189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 19/37] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:02.537865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa9aac-b982-41f4-985b-91859a6d1111 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:02.607908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:02.610473 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 19/38] 10.222.0.42 () {60 vars in 1131 bytes} [Fri May 27 16:06:02 2022] POST /identity/v3/roles => generated 225 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:02.612088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa1a06-d3b4-4b4b-a3d1-2bb8d8696c2d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.612392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa1a06-d3b4-4b4b-a3d1-2bb8d8696c2d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.612680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa1a06-d3b4-4b4b-a3d1-2bb8d8696c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.631986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6efa1a06-d3b4-4b4b-a3d1-2bb8d8696c2d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:02.634977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6efa1a06-d3b4-4b4b-a3d1-2bb8d8696c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:02.653793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa9aac-b982-41f4-985b-91859a6d1111 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.695115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.695917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.696205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.696760 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.697838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.699484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299fe31-e03b-4947-934d-8de511d3c6d5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.706158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 20/39] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:02.706158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2361-4ed4-4734-b207-ff5a9019768e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.706158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2361-4ed4-4734-b207-ff5a9019768e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.706158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db2361-4ed4-4734-b207-ff5a9019768e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.706158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 21/40] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:02.707995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db2361-4ed4-4734-b207-ff5a9019768e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:02.725801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:02.727105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.727402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.727685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.752431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:02.753745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:02.830509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db2361-4ed4-4734-b207-ff5a9019768e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.832035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:02.832500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:02.864886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:02.873672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.875329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.875769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.876233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] PATH_INFO: `/v3/domains/default/users/e387d1cbdd3947bcb9659065cbae32c0/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.878655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e387d1cbdd3947bcb9659065cbae32c0, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.880294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.890903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:02.939421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8151a9-b348-4a62-bfa7-85ad39328d9a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:02.973222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:02.973922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:02.974219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:02.974505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:02.975252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:02.976728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:02.988715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Fri May 27 16:06:02 2022] PUT /identity/v3/domains/default/users/e387d1cbdd3947bcb9659065cbae32c0/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:06:02.990091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:03.072522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1131 bytes} [Fri May 27 16:06:02 2022] POST /identity/v3/roles => generated 223 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:03.106323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959a5-590b-4d67-8123-70b3ae134e9b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:03.148862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:03.149984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:03.150425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:03.150846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:03.152431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:03.154398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa73505-fe63-45b9-9b63-389e1849a2ef None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:03.156439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:03.162553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:03.279578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbfad3-5fc5-474a-b8d2-0524a9f8cc78 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:03.291978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:03.316678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:03.319679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:03.319679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:03.319679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:03.319679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:03.320164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c67e5-c643-4ee9-bcdd-6cdf1f9e0062 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:03.394218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:06:03 2022] POST /identity/v3/projects => generated 312 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:05.203877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8765-9eca-4c4a-87c6-8116f88649d4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.204363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8765-9eca-4c4a-87c6-8116f88649d4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.204363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c8765-9eca-4c4a-87c6-8116f88649d4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.205785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:05.211423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.211812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.211994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.229049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:05.230260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:05.295549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:05.319155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c8765-9eca-4c4a-87c6-8116f88649d4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.379354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:05.425642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c8765-9eca-4c4a-87c6-8116f88649d4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.461702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:05.462416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.462703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.462984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.472827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-9e6075ea-d53b-42a5-9d15-d52ba2de8564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:05.480417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:05.481376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:05.482137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:05.488110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.576549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:05.585279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41282a28-4911-4f1e-9669-fe3235a58ecc None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.602947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fddb49-4a49-48a3-bb6f-c61f9103bd5f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.603246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fddb49-4a49-48a3-bb6f-c61f9103bd5f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.603536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fddb49-4a49-48a3-bb6f-c61f9103bd5f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.604924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 26/50] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:05.611220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.611510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.611790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.620173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:05.620847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.621142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.621501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.622243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:05.622561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:05.623977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:05.628405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:05.629950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:05.636302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:05.641805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.698506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 27/52] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:05.722134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.746816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c67f3-0c2b-4fe7-b882-411dbf5de2e6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.771233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:05.785454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:05.786338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.786630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.786918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.788153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:05.789654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:05.791477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:05.798013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.822335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:05.824701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62bcc1-0672-40f5-8112-df5e7339e319 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.859411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:05.860210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.860499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.860783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.861992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:05.863495 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:05.865144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 28/54] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:05.870613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.896420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6e784-acca-4af4-beb6-5bbb22ac27f9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.933746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:05.934252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:05.934550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:05.934839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:05.936041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:05.938030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:05.939484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:05.944919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:05.972740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dca4c3-d89f-4057-b7dc-16e388fae5bd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:05.989866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:06.006253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:06.006918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:06.007208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:06.007490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:06.008208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:06.009725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91584a-0502-4d38-93e8-83c8474d7455 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:06.034467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:06.044064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb267b3b-78a3-4d41-b78d-f4481ac965bb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:06.078424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:06.079084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:06.079375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:06.079659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:06.080507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.domain.id=default, user.id=e387d1cbdd3947bcb9659065cbae32c0) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:06.080832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:06.082513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8c5e0-c298-40f2-be73-ad8cc2d98104 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:06.085938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 29/56] 10.222.0.42 () {60 vars in 1140 bytes} [Fri May 27 16:06:05 2022] POST /identity/v3/projects => generated 298 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:06:06.088470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1322 bytes} [Fri May 27 16:06:05 2022] GET /identity/v3/role_assignments?user.id=e387d1cbdd3947bcb9659065cbae32c0&scope.domain.id=default&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:08.305558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185893-d27e-4a92-a94f-9191fc2d6a4b None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:08.306108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185893-d27e-4a92-a94f-9191fc2d6a4b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:08.306108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48185893-d27e-4a92-a94f-9191fc2d6a4b None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:08.307428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:08.313033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:08.313417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:08.313639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:08.330949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:08.332140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:08.395334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:08.419027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48185893-d27e-4a92-a94f-9191fc2d6a4b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:08.519685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48185893-d27e-4a92-a94f-9191fc2d6a4b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:08.552020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:08.552799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:08.553086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:08.553384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:08.554550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:08.556053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7abf91-9ab5-4df5-b30e-463672788e9e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:08.557623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:08.563798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:08.644097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:08.659776 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10316d-a1f2-4fa4-8635-5d62323634b6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:08.693300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:08.693999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:08.694292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:08.694576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:08.695294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:08.696750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815bc9e-f60d-4b4f-8d35-f894fc4f1cee None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:08.807669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:08 2022] POST /identity/v3/users => generated 275 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:12.281123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f34263-6bd5-4ed7-b9a9-c8e8032d7e3b None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.282146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f34263-6bd5-4ed7-b9a9-c8e8032d7e3b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.282642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f34263-6bd5-4ed7-b9a9-c8e8032d7e3b None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.284479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:12.290437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab228-2a4a-454e-8560-9bfb255dae33 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.291297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab228-2a4a-454e-8560-9bfb255dae33 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.292032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab228-2a4a-454e-8560-9bfb255dae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.309844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4c3ab228-2a4a-454e-8560-9bfb255dae33 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:12.311566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4c3ab228-2a4a-454e-8560-9bfb255dae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:12.343502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764274fa-f265-4ecb-a4b5-33b96c65b611 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.344120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764274fa-f265-4ecb-a4b5-33b96c65b611 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.344626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764274fa-f265-4ecb-a4b5-33b96c65b611 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.346252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 33/63] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:12.353186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.353759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.354262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.378431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:12.379656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:12.404101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 31/64] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:12.413916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705334d-bbe9-473b-a393-905aae5bfb49 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.414532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705334d-bbe9-473b-a393-905aae5bfb49 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.415017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705334d-bbe9-473b-a393-905aae5bfb49 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.416651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 32/65] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:12.426722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.427277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.427784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.433548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:12.450564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:12.452679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:12.472768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 34/66] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:12.474243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.551998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:12.565386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 33/67] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:12.565386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.619045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fad0f-bde8-4f84-a04e-b454fb132c86 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:12.657299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:12.658627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.658915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.659339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.672769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:12.679928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a590-ca12-4532-997b-15c6f8ff23bc None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:12.680764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:12.681637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:12.682384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:12.685036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 35/68] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:12.686580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.724458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:12.725284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.725597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.725885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.727643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:12.739635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:12.751071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:12.751980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:12.752727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:12.753295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:12.754200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.802109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1fcd4-2d98-4db6-8cc9-2a55569e6c68 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:12.814401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:12.836746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:12.837620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.837907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.838198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.848747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:12.854712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11049-770a-4436-85d7-07a42348b4d6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3947c19f-527c-44a7-90ed-212868849d3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:12.856310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:12.857291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:12.859064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 36/70] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:12.859843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628236-a84b-4907-b674-8d9add203485 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.859843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628236-a84b-4907-b674-8d9add203485 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.859843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4628236-a84b-4907-b674-8d9add203485 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.861299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 37/71] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:12.862221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a172-e9d3-4e5f-af3e-eb6bf25f2929 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.862712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a172-e9d3-4e5f-af3e-eb6bf25f2929 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.862859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a172-e9d3-4e5f-af3e-eb6bf25f2929 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.867930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 38/72] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:12.867930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a172-e9d3-4e5f-af3e-eb6bf25f2929 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.898087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:12.898792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:12.899082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:12.899363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:12.900119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:12.900426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:12.901891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:12.910020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 35/73] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:12.910805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:12.955836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:12.970428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a172-e9d3-4e5f-af3e-eb6bf25f2929 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.003681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.004372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.004899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.005358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.006271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:13.006731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.008472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022d9cc-b244-4e79-8511-ed007e3a2911 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.013355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60104d4-d7bf-4ba4-abc7-99baba396f5b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.017406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 39/74] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.018615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.018910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.019194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.035668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:13.036850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:13.037206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.048945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.049657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.049955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.050241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.051006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:13.051324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.052765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c9df4-f422-40f6-a9a2-78245fe45f56 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.061141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 36/75] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.062457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.062761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.063050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.079574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:13.080793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:13.109878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 40/76] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:13.110730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.160900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 37/77] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:13.161780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.194918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.214230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce666d-42b8-47c3-bc06-45807cde2268 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.249285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.250189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.250477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.251127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.251964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.253464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.255079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 41/78] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.255833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.279469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d4112-06c0-4adb-8463-440c54668c9a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.294309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.315287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.316100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.316393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.316679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.330722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.332213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.334338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 38/79] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.335169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.355737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e6c6c-305d-4fea-8306-2d1564dcea7f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.389953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.390920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.391221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.391499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.392693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.394173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.395787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 42/80] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.396546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.415160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.433407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77047b7-5c9b-4763-ad1b-88f135790564 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.467048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.467937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.468156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.468438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.478353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:13.486574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:13.487446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:13.488194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 39/81] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:13.488801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.502619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed10250-570a-4dc1-b763-2eccf5ecf8a9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.520000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.535838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.536631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.536917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.539726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.539726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.539928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.542008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 43/82] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.542674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.588529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c56566-ab93-4d9f-9f5e-1e567978e225 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.621929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.622578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.622862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.623151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.626293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.641078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0ffa4-18d1-4e05-ae39-c04a38886b99 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.675676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.676329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.676620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.676904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.690586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.692095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.720636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.744249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.745774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.788876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 40/83] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:13.789767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.828664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.845085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 44/84] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:13.845873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:13.889779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba204213-c68c-4bd6-893f-58e667b4d3fb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.918647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:13.923651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.924308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.924599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.924885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.947619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60365f18-c675-470c-b8ad-f0d0deb3e258 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:13.981260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:13.981962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:13.982255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:13.982537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:13.983875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:13.984191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.985629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.992306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:13.993581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 45/85] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:13.993836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:13.994351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.042151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.092704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 41/86] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:14.093652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.097513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124859b2-32d1-4e5d-b7d5-a3aeedb2f281 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.125088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.131888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.132564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.134579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.134579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.134579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.135514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.193160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27005f-c42d-4a21-a3b0-af71b91107e6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.216274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 46/87] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:13 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:14.217113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.230001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.231243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.231542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.231899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.233844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.235493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.237707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 42/88] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.238559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.347398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8b2a4-8833-44f7-8366-f304f77915d5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.348568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.350384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.351568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1774d-3d87-4b3c-a4fe-32705622c35c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.387275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.388155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.388450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.388737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.390717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.392220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.393562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.398875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 43/89] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.398956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.399022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.399086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.399086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.400058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.401851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.404845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 47/90] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.405666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.517477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0e010-cf61-4668-b452-59bdbb241cde None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.525796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd40e3-e687-477e-ac47-3511970b9140 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.561278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.562497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.562796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.563082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.564318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.565836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.567560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 44/91] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.568411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.569658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.570572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.571027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.571472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.572504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=223989f934384b859e00e27e484a6892, user.id=e387d1cbdd3947bcb9659065cbae32c0) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:14.572993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.574692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.584871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 48/92] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/role_assignments?user.id=e387d1cbdd3947bcb9659065cbae32c0&scope.project.id=223989f934384b859e00e27e484a6892&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.587528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.615618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.630656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:14.724823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba18caa-d62e-4a9e-9960-cebb82382a96 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.736699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadd1ca-3ce1-4d01-88d3-57ca0c3f277a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:14.795529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.796328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.796609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.796769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.802934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:14.803611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:14.803898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:14.804175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:14.805018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=223989f934384b859e00e27e484a6892, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:14.805353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.806855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.821144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 49/93] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=223989f934384b859e00e27e484a6892&role.id=4d146a1b44df454bbbc17abada36747a => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:14.822042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:14.916192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:14.918236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:14.979953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:15.022014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:15.028146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f73658-5760-4921-ae55-ac124aa8502a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:15.099122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:15.099785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:15.100073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:15.100353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:15.101193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=223989f934384b859e00e27e484a6892, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:15.101532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:15.103001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:15.107347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 45/94] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:15.111494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 50/95] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:14 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=223989f934384b859e00e27e484a6892&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:15.113855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:15.262092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851854-c295-4fd6-b104-576ba4a9ad0f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:15.308898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:15.309824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:15.310110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:15.310389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] PATH_INFO: `/v3/projects/6ab5cd17a19e470c8038cc83d233848c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:15.318221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] RBAC: Authorizing `identity:get_project(project_id=6ab5cd17a19e470c8038cc83d233848c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:15.319747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e83b8-0f5e-40b5-9e8f-1e93a258f28c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:15.321908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 46/96] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:15 2022] GET /identity/v3/projects/6ab5cd17a19e470c8038cc83d233848c => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:15.327499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:15.454190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088680-85c5-469b-9564-eb66049e29f0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:15.491478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:15.492134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:15.492425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:15.492703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:15.493560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=6ab5cd17a19e470c8038cc83d233848c, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:15.493880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:15.495337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6a4ba-5386-4391-b212-7671b79d86db None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:15.502097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 51/97] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:15 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=6ab5cd17a19e470c8038cc83d233848c&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:17.375952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8286-a548-424a-8a38-8b0816cfabb1 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.376459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8286-a548-424a-8a38-8b0816cfabb1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.376633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff8286-a548-424a-8a38-8b0816cfabb1 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:17.378060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 47/98] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:17.383683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.384126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.384312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:17.416811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:17.418040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:17.444906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:17.503932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 52/99] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:17.541252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8286-a548-424a-8a38-8b0816cfabb1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:17.546085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:17.665565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff8286-a548-424a-8a38-8b0816cfabb1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:17.719066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:17.719880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.720170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.720453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:17.721691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:17.723170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f1704-9d17-4d12-b22a-60dc67dc7e7b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:17.724957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 48/100] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:17.730711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:17.885313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aae6e-5fc8-4f0b-a7b3-7e45c4994e50 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:17.937792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:17.938476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.938765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.939045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:17.939769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:17.941226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1f79b-8acd-44e2-b5cc-a181ec5d1a49 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:17.968757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f158fe-8f29-4dbb-8690-329cc2d8ea0d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.969070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f158fe-8f29-4dbb-8690-329cc2d8ea0d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.969319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f158fe-8f29-4dbb-8690-329cc2d8ea0d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:17.970912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 49/101] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:17.975932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b388e-dbfd-49e2-a281-7089309c4255 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:17.976260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b388e-dbfd-49e2-a281-7089309c4255 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:17.976547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b388e-dbfd-49e2-a281-7089309c4255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.002202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e47b388e-dbfd-49e2-a281-7089309c4255 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:18.003445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e47b388e-dbfd-49e2-a281-7089309c4255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:18.038072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 53/102] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:06:17 2022] POST /identity/v3/projects => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:18.101785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 50/103] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:18.113393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ed0d3-5797-4053-875c-42aa151bc1fe None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.113694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ed0d3-5797-4053-875c-42aa151bc1fe None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.113979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ed0d3-5797-4053-875c-42aa151bc1fe None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.115418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 54/104] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:18.120821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.121119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.121434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.127769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.132354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a1f2-56d4-47d3-94bd-327abf6463f6 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.139596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a1f2-56d4-47d3-94bd-327abf6463f6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.139596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a1f2-56d4-47d3-94bd-327abf6463f6 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.139596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 55/105] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:18.140800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.141159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.141470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.161166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:18.165662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:18.169039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:18.170408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:18.180300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.305755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 56/106] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:18.306567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.307388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 51/107] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:18.341198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.439857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77da5-3a4c-4420-a392-ce296d2d114e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:18.453613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9695b-cba4-4bb8-a588-fbbe2c17b41b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:18.484952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.486443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:18.487153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.487539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.487830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.492690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:18.493889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.494387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.494811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.500613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.500949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:18.509469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:18.510336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:18.511097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 57/108] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:18.511720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.520871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-7d014d12-1a7d-48d8-8ea7-3a779d686dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:18.525257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:18.526408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:18.527366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:18.528933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 52/109] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:18.531343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9c6b-c414-44c2-b907-25255fca3582 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.531343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9c6b-c414-44c2-b907-25255fca3582 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.531343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd9c6b-c414-44c2-b907-25255fca3582 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.532071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 53/110] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:18.533625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd9c6b-c414-44c2-b907-25255fca3582 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.645443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12541a63-e377-403c-9b9f-30f4e50880af None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:18.665685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd9c6b-c414-44c2-b907-25255fca3582 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:18.687363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:18.688169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.688463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.688751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.703357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:18.704629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:18.705583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.706104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.706581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.707567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:18.708060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:18.709730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146796-5f56-4222-97bb-6b1ac14dda1f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:18.711075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:18.711928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:18.712745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 58/111] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:18.713502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.726960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:18.728627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.729108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.729756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.748935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:18.750807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:18.809300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.814825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:18.831698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93196d-1f1e-4569-a06a-900fa80543b0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:18.836804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 55/113] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:18.842423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.870647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:18.871504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:18.871899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:18.872607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:18.874209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:18.874782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:18.876364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:18.887987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 59/114] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:18.906596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:18.962967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea3f19-7358-4293-892c-8cc8bce7f6a9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.006718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.007401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.007696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.007982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.008959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:19.009280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.011058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.025425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 56/115] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.025425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.033448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035571e-3e6b-410f-8ad9-c829169c26dc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.058614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.079351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.080471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.080782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.081481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.082046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.083453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.084981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.087467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 60/116] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:18 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.088302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.141244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a660b-1606-4cd6-887d-54f43f431ab3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.180296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.181144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.181465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.181752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.194667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:19.195912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2d79-c7a2-4178-8cf6-c94f61dc8b0b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:19.203312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:19.204169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 57/117] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:19.210264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.240248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.241125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.241448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.241742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.242938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.244405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec7fb0-f6c0-47dd-8582-d210a178f4ce None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.246411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 61/118] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.247682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c5b3-bc63-44bb-b4bb-8b7a9cae65e8 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.247985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c5b3-bc63-44bb-b4bb-8b7a9cae65e8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.248903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c5b3-bc63-44bb-b4bb-8b7a9cae65e8 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.250387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 62/119] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:19.255208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861c5b3-bc63-44bb-b4bb-8b7a9cae65e8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.314756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.322677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bf8b-2784-474b-ab18-dc0e1c1a42eb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.331632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.359298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.359600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861c5b3-bc63-44bb-b4bb-8b7a9cae65e8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.360175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.360467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.360751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] PATH_INFO: `/v3/users/599fadf17127442e90896add7cae1ca4` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.374819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] RBAC: Authorizing `identity:get_user(user_id=599fadf17127442e90896add7cae1ca4)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.376358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.382648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 58/120] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/users/599fadf17127442e90896add7cae1ca4 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.383510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.397127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.398040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.398335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.398608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.399794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.401261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.402993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 63/121] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.403747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.498660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd308865-9d61-4666-997d-7495344d9d26 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.509265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc596c8-e4d9-4193-866b-3339b5af45a8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.534755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.537895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.538573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.538856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.539132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.539883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:19.540195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.541887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525dc19-8ab1-4b16-89c5-5a9b7d362310 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.546793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.547656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.547952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.548231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.549528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.551011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.551887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 59/122] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.552836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 64/123] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:19.553719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.554167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.554601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.558015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.573191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:19.574428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:19.599783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.652983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 60/124] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:19.654095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.667144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb17eb3-6a56-427d-9049-003469f85105 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.706420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.707101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.707396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.707680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.768334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11d0a-255f-4fb0-87be-3e01aba4efb2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:19.769227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.779828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.781470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.805445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:19.806340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:19.806636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:19.806925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/users/460698a7b6d64938b66f2e216b2e86c3/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:19.809188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, user_id=460698a7b6d64938b66f2e216b2e86c3, role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:19.811420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:19.882077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 65/125] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:19.882918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.920684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 61/126] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:19 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/users/460698a7b6d64938b66f2e216b2e86c3/roles/4d146a1b44df454bbbc17abada36747a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:19.921945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:19.952861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:19.992246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:20.004680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aebf51-aa1d-4eb1-a809-fa61f2844ea4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.045845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.046976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.047331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.047722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.049095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.050822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.052588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 66/127] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:19 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:20.053907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.071327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decac4-2f15-453a-be43-7a634be9975f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.121296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.122881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.123928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.124631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/users/e387d1cbdd3947bcb9659065cbae32c0/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.127171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, user_id=e387d1cbdd3947bcb9659065cbae32c0, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.128675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.175860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206d44b-e414-47a3-adc4-1029b0dec598 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.211282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.212093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.212382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.212664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.223271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:20.231046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:20.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:20.233316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 67/128] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:20.234530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.259429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:20.259786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 62/129] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:19 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/users/e387d1cbdd3947bcb9659065cbae32c0/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:20.261254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.369728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:20.397955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe642b-d21d-44bc-8f4d-387576e40b55 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.446844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.447822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.448115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.448400 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.454994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.456599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.458457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 63/130] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:20.459277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.511670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a526d-93b6-4943-b26d-7b204a3aa67b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.579341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.580216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.580510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.580795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.582146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.584426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.587236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 68/131] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:20.588020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.627397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3a19-a0e9-490f-94ba-eb131b7529b8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.691237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.691926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.694441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.694751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.695550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:20.695870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.703422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.729442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 64/132] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:20.730417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.744567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:20.746927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:20.760328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137268ec-8d6c-48cd-8c5e-ad2f67cbafcc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.804446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.805541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.805838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.806179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/users/460698a7b6d64938b66f2e216b2e86c3/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.808950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, user_id=460698a7b6d64938b66f2e216b2e86c3, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.811121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0f5f2-61b2-4281-9c9e-b788e67d76c2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.861607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31db43a-a88a-4897-9ee4-33d25e82233a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:20.877975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 69/133] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:20 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/users/460698a7b6d64938b66f2e216b2e86c3/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:20.879628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b3d50-c9ad-4c9a-97f0-988c6f40abbf None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.880120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b3d50-c9ad-4c9a-97f0-988c6f40abbf None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.880617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b3d50-c9ad-4c9a-97f0-988c6f40abbf None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.882338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 70/134] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:20.883331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b3d50-c9ad-4c9a-97f0-988c6f40abbf None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:20.953369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:20.953369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.953369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.953369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:20.953771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, scope.project.id=223989f934384b859e00e27e484a6892, user.id=599fadf17127442e90896add7cae1ca4) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:20.953771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:20.953771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c79b95-09da-49eb-badb-41af9ba75c93 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:20.962742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 65/135] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/role_assignments?user.id=599fadf17127442e90896add7cae1ca4&scope.project.id=223989f934384b859e00e27e484a6892&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:20.964419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:20.964932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:20.965436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.003471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:21.006107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:21.134287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b3d50-c9ad-4c9a-97f0-988c6f40abbf None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:21.153815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:21.164710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 66/136] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:21.191770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:21.192728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:21.193026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:21.193314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.194829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:21.196426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:21.198446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 71/137] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:20 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:21.205537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:21.217492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:21.261124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:21.411153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d5b65-594e-4c3a-8b0d-130357ac4683 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:21.434870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce671b-77c1-4cee-9a64-c8daf47fae44 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:21.473666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:21.474531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:21.474817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:21.475095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] PATH_INFO: `/v3/projects/6ab5cd17a19e470c8038cc83d233848c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.476390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=6ab5cd17a19e470c8038cc83d233848c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:21.477893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:21.479583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 67/138] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:21 2022] GET /identity/v3/projects/6ab5cd17a19e470c8038cc83d233848c => generated 312 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:21.485906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:21.497638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:21.499163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:21.499454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:21.499735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.502487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:21.503950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:21.506840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 72/139] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:21.510915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:21.647831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:21.659167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:21.666447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0188-a434-40e7-b2b0-12c4662615f1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:21.735230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:21.735934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:21.736221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:21.736500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.737224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:21.738885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9c998-ef52-4994-b61d-98440b45c02f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:21.742345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2746ec7-5433-4300-ae8d-a5a0ea178ed3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:21.803418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:21.804720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:21.805225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:21.805775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] PATH_INFO: `/v3/projects/6ab5cd17a19e470c8038cc83d233848c/users/460698a7b6d64938b66f2e216b2e86c3/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:21.809918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ab5cd17a19e470c8038cc83d233848c, user_id=460698a7b6d64938b66f2e216b2e86c3, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:21.811751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b663832-2264-4cfc-ab15-65154963a123 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:21.865886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 73/140] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:21 2022] POST /identity/v3/users => generated 283 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:06:21.926223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 68/141] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:21 2022] PUT /identity/v3/projects/6ab5cd17a19e470c8038cc83d233848c/users/460698a7b6d64938b66f2e216b2e86c3/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:06:22.905513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57188-7515-4ce1-ba74-1df48bc9ce90 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:22.906099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57188-7515-4ce1-ba74-1df48bc9ce90 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:22.906099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57188-7515-4ce1-ba74-1df48bc9ce90 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:22.907610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 74/142] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:22.913059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5a62-f2d8-419a-a7db-85da73fa0abe None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:22.913910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5a62-f2d8-419a-a7db-85da73fa0abe None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:22.914386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c5a62-f2d8-419a-a7db-85da73fa0abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:22.938356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-135c5a62-f2d8-419a-a7db-85da73fa0abe None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:22.939566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-135c5a62-f2d8-419a-a7db-85da73fa0abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:23.020728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:23.064423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57188-7515-4ce1-ba74-1df48bc9ce90 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.111441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022e27-37d4-4a6e-85a3-50aa03dc47e4 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.111736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022e27-37d4-4a6e-85a3-50aa03dc47e4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.112024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce022e27-37d4-4a6e-85a3-50aa03dc47e4 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.113545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 70/144] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:23.119934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.120224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.120513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.155230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:23.156684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:23.226907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57188-7515-4ce1-ba74-1df48bc9ce90 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:23.293729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:23.294380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:23.295390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.295728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.296004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.321022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:23.336082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:23.336976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:23.337766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 75/145] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:23.339084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:23.344027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 71/146] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 16:06:23.344974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.385222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.514052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbf226-09db-4211-9ee7-36f93a436968 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:23.583104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:23.583903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.584193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.584470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.585226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:23.585560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:23.586956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:23.603173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 76/147] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:23.610610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.649224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d3866-5e89-4b60-8df2-d86f3d9228a0 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:23.708549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:23.738234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:23.739434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.739914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.740386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.769418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b2ebe-df6c-4f06-acd7-3dd3668a292d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:23.770673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:23.782767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:23.783652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:23.784401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:23.787086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 72/148] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:23.793110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.817951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:23.818883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:23.819197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:23.819450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:23.820948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:23.822502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:23.824350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 77/149] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:23.830255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:23.908138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:24.033054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fa2a8-82d4-4ff5-a304-2ee0c5af6688 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:24.079665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28e41d-abef-49d4-869d-0c0ae9fbcdaa None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:24.130954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:24.131643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.131930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.132208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.132972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:24.133282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:24.134730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07fcfb-0c9d-48a2-9f5f-baa9e2076e83 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:24.145588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:24.156405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 73/150] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:24.157612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9499b-b7a6-49db-98e7-bc47a8d2b429 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.157946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9499b-b7a6-49db-98e7-bc47a8d2b429 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.158178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9499b-b7a6-49db-98e7-bc47a8d2b429 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.159403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 74/151] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:24.160167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:24.161132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.161232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.161604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.166528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9499b-b7a6-49db-98e7-bc47a8d2b429 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:24.317361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:24.318813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc9a4d-86fc-4c6a-9d2b-0d0b70f75fb0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:24.406483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9499b-b7a6-49db-98e7-bc47a8d2b429 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:24.428635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:24.478403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:24.479680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.480933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.484222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] PATH_INFO: `/v3/users/e387d1cbdd3947bcb9659065cbae32c0` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.485719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] RBAC: Authorizing `identity:get_user(user_id=e387d1cbdd3947bcb9659065cbae32c0)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:24.487337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476491-8649-4b97-acc5-efc84e615b0a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:24.490472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:24 2022] GET /identity/v3/users/e387d1cbdd3947bcb9659065cbae32c0 => generated 247 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:24.493430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49ae2-df44-41f9-979f-6df14a3a74cc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.494658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49ae2-df44-41f9-979f-6df14a3a74cc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.495695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de49ae2-df44-41f9-979f-6df14a3a74cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.509973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 78/153] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:24.519678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a0e-9ef7-44b1-9ce3-42dd8de91d7f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.520157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a0e-9ef7-44b1-9ce3-42dd8de91d7f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.520595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a0e-9ef7-44b1-9ce3-42dd8de91d7f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.522128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 79/154] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:24.523096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117a0e-9ef7-44b1-9ce3-42dd8de91d7f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:24.536192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-8de49ae2-df44-41f9-979f-6df14a3a74cc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:24.538000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8de49ae2-df44-41f9-979f-6df14a3a74cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:24.579382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:24.711580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 76/155] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:24.713183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.714790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.715425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.760186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:24.769200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:24.787074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117a0e-9ef7-44b1-9ce3-42dd8de91d7f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:24.866113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:24.867205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:24.867760 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:24.869009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:24.936842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 77/156] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:24.936842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.036385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.037921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:25.038466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.105991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.133534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd9ec5-93e2-4066-98f9-f5d081cbc511 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.210063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.211110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.211496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.211876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.213204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.214298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 80/157] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:25.215211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:25.215647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.230324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 78/158] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:24 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:25.231323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.325023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce814c7-cd8c-41d9-aa5a-8c1467f68c36 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.357407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.364425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.365729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.366199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.378126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.379584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:25.388150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:25.389316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:25.390096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:25.392715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 81/159] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:25.394079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.472247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ddeb5-4f9f-4888-a8b2-58d6cc908608 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.484272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.510927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a1341-a154-4a2e-b78c-b2d5c658f1c0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.553914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.555256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.555766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.556261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.557803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.559509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:25.561761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 82/160] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:25.562862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.570181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.571023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.571311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.571588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.602283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:25.610985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:25.611856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:25.612632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:25.617774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 79/161] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:25.618626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.634181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.682873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2ef3-fbfb-4b35-aefe-5c698fbc8aa4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.726135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.727164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.727669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.728140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.729180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=223989f934384b859e00e27e484a6892, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:25.729735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.731555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:25.738993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 83/162] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=223989f934384b859e00e27e484a6892&role.id=4d146a1b44df454bbbc17abada36747a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:25.740197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.859560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf54-323a-4839-8aa3-85784a262634 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.861457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a0be9-328e-43f2-8df8-7c2424453e1e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:25.935766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:25.960731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.961433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.961720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.961995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.962909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=223989f934384b859e00e27e484a6892, user.id=e387d1cbdd3947bcb9659065cbae32c0) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:25.963223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.964735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:25.987483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 84/163] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/role_assignments?user.id=e387d1cbdd3947bcb9659065cbae32c0&scope.project.id=223989f934384b859e00e27e484a6892&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:25.988354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:25.994760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:25.995457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:25.995741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:25.996019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:25.996778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:25.997088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:25.999479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cc48d-c7b3-4546-b474-f40414e18ef9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.027498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 80/164] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.029402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5032b-359b-44cc-947c-b4322b642636 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.029709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5032b-359b-44cc-947c-b4322b642636 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.029989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5032b-359b-44cc-947c-b4322b642636 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.031577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 81/165] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:26.036840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5032b-359b-44cc-947c-b4322b642636 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.134516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.147127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d107cc-3369-487b-973d-1b22c49726a8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.190617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.191590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.192031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.192499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.193515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:26.193980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.195626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b753adc-ba0b-4dbc-9bf9-177e730adba0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.206910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 85/166] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.208767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.209208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.209705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.228998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:26.230650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:26.253381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5032b-359b-44cc-947c-b4322b642636 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.284485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.296004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.299984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.299984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.299984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.299984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.300919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.302804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 82/167] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.310919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.313372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 86/168] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:26.314612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.417582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.427206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775f136-a610-4d0a-ad58-b25ebb156bae None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.471019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf82c0a-4042-4c56-8742-8edabb0823ef None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.504008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.505261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.505685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.505974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] PATH_INFO: `/v3/users/599fadf17127442e90896add7cae1ca4` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.508052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] RBAC: Authorizing `identity:get_user(user_id=599fadf17127442e90896add7cae1ca4)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.509606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.511289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 83/169] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/users/599fadf17127442e90896add7cae1ca4 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.512126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.528516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.529998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.530538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.531002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.584298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.612440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.620271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f9f93-c26f-422e-b5b3-b2ea2cdfc208 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.628478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991c751-9a34-4330-a750-c02693004c61 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.668540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.669807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.670547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.671146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.688813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:26.697296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:26.698189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:26.698942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:26.701496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:26.702577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.731377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 87/171] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:26.733636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc574b-b872-493d-b603-e301ca11fd87 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.734045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc574b-b872-493d-b603-e301ca11fd87 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.734334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc574b-b872-493d-b603-e301ca11fd87 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.734848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.736089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 88/172] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:26.736733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc574b-b872-493d-b603-e301ca11fd87 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.829270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d04efa-b6a0-401f-a8e8-2cd4a339e533 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.835393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:26.861990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc574b-b872-493d-b603-e301ca11fd87 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:26.870342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.871646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.872133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.872674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.874769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.876585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d389d-65de-4592-ab3a-7f22be3c4f12 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.878650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 85/173] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.880294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.880784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.881260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.900076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:26.901649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:26.902249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:26.902974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:26.903265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:26.903600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:26.905593 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:26.905960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:26.907460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:26.917617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 89/174] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:26.918555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:26.985801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 86/175] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:26.986623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.012108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.048777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f05b48-beb2-4f87-8af5-7c336338b4f3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.089478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.090641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.090924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.091198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.092972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.094185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5eaaa-5f7e-48b7-a1dd-721d5b2bab7c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.094458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.096140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 90/176] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:26 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:27.096898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.112903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.130129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.131006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.131290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.131564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/users/599fadf17127442e90896add7cae1ca4/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.133518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, user_id=599fadf17127442e90896add7cae1ca4, role_id=af4dcdb2cb0f4bfe9db96cddc9eac04d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.135010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c1af-5567-41cf-b87b-cb7bfbff694f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.198862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc363-d55c-490e-904d-93de447c02c9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.209524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 87/177] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:26 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/users/599fadf17127442e90896add7cae1ca4/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:27.210808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21150f-a7cd-4394-8e1d-960f8d9d6e05 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.211107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21150f-a7cd-4394-8e1d-960f8d9d6e05 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.211390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21150f-a7cd-4394-8e1d-960f8d9d6e05 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.212831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 88/178] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:27.213442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21150f-a7cd-4394-8e1d-960f8d9d6e05 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.241043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.241886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.242169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.242444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] PATH_INFO: `/v3/users/460698a7b6d64938b66f2e216b2e86c3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.243894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] RBAC: Authorizing `identity:get_user(user_id=460698a7b6d64938b66f2e216b2e86c3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.245419 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.247200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 91/179] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/users/460698a7b6d64938b66f2e216b2e86c3 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:27.248065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.320027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.339906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21150f-a7cd-4394-8e1d-960f8d9d6e05 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.345927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.375205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013112-0d00-441e-962a-fa69efe6d766 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.380554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.381402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.381703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.381978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.396617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1dc36d84-789c-4ecf-b20d-cd136da22b67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:27.404510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:27.405409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:27.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 89/180] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:27.407446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f7de-8a99-43d1-93c6-4d1168059902 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.407446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f7de-8a99-43d1-93c6-4d1168059902 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.407777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f7de-8a99-43d1-93c6-4d1168059902 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.411090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 90/181] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:27.411555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.411848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.412128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.417491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.418159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.418445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.418721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.419558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=223989f934384b859e00e27e484a6892, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:27.419878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.421426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.432979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 92/182] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=223989f934384b859e00e27e484a6892&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:27.432979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.433243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:27.433243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:27.526801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 91/183] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:27.527728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.595946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a446-c936-4ce6-ac9d-cb5a58cf6101 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.628893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.645835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.646787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.647273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.647703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.661153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.667275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8507c-ae02-4772-8f87-e4956073c5e4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.709691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.710372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.710659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.710937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.711703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:27.712014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.713457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a2f4-355d-4965-a7d1-920b7f9262d7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.725407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 92/184] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:27.729538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.729985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.730276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.739986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.743851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.753395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:27.754599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:27.825323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 93/185] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:27.826788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.860433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 93/186] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:27.861375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:27.900780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.903326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:27.947491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d7d99-855c-4d25-b277-8c42eb4160f3 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.968062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7cc8-dec4-442c-aa2e-51e09976902f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:27.988343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:27.989415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:27.989746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:27.990039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:27.991354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:27.992830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:27.999033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 94/187] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:27.999033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.014833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.015714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.015996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.016276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.032902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.034398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.036319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 94/188] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:28.036984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.106237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec3c8-f724-450e-8b09-92e874349109 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.136485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.143497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.143643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.144377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.150513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.150513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.150823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f23310-cef0-4351-80e1-c23451cf6781 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.158034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:28.166374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:28.167245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:28.168008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:28.172689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 95/189] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:28.173558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.183685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.184559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.184848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.186269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] PATH_INFO: `/v3/projects/6ab5cd17a19e470c8038cc83d233848c` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.187525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] RBAC: Authorizing `identity:get_project(project_id=6ab5cd17a19e470c8038cc83d233848c)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.188999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.195787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 95/190] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/projects/6ab5cd17a19e470c8038cc83d233848c => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:28.195787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.296486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9e5a-5a0a-470a-a526-0228863a3f3e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.312397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514322a5-0413-4485-9e37-0d62aa75e998 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.334181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.334860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.335147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.335428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.336141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.337631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.351728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.357068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.357068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.357068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.393356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.429265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.430852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.451943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 96/191] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:28 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:28.462136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.534870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.595367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 96/192] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:28.597211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.607740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd2e9-afdf-455e-a47d-9a38ed325e4c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.660174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.661214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.662240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.662711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.664446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:28.664971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.667296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.686610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:28.688558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.702681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.754604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16f090-567c-444e-a731-ca3879438fff None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.818944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.819620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.819911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.820197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.821113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=6ab5cd17a19e470c8038cc83d233848c, user.id=460698a7b6d64938b66f2e216b2e86c3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:28.821532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.823014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.832374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 97/194] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/role_assignments?user.id=460698a7b6d64938b66f2e216b2e86c3&scope.project.id=6ab5cd17a19e470c8038cc83d233848c&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:28.833029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.847942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aef73-7490-4b51-a96e-dcb24855ca0d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:28.865844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:28.899968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:28.901438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:28.902391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:28.903145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:28.906188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:28.909544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:28.912358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 98/195] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:28.921104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:28.986124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2170cf-ff80-46cb-a627-d178a99077e2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:29.026507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:29.036886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:29.037989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.046092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.046092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.046092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:29.046092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:29.046092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:29.049816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:29.061037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ab5df-c86d-4dbb-8303-5b3159dd0088 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:29.106764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:29.108525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.109009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.109552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.110769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=e57981725c734fe3b61ccf7487f2ce7b) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:29.111309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:29.113131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:29.120213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 99/197] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:28 2022] GET /identity/v3/role_assignments?user.id=e57981725c734fe3b61ccf7487f2ce7b&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:29.198552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea7ead-a35c-4ba6-951c-29b14fda7725 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:29.258969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:29.259640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.259927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.260209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.375559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:29.381580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:29.383403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:29.503567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 99/198] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 16:06:29.508883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:29.573690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:29.653407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7b023-2e79-4276-81c4-70afa9cfe056 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:29.696839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:29.697789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.698081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.698371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.699728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:29.701495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf05c4-add7-46f5-8fc2-801dcc93c3be None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:29.703367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 100/199] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:29 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:29.709510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:29.831393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:29.835786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bbff4-8794-43ea-be1d-d680e47ccd05 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:29.877498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:29.878171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.878469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.878752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.879599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=e57981725c734fe3b61ccf7487f2ce7b) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:29.879917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:29.881568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702da478-1a29-40be-a466-4571f1fbe280 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:29.888301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4513-eeef-47c0-8b12-1066c62e8178 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.888661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4513-eeef-47c0-8b12-1066c62e8178 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.889053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4513-eeef-47c0-8b12-1066c62e8178 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.890469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 100/200] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:29 2022] GET /identity/v3/role_assignments?user.id=e57981725c734fe3b61ccf7487f2ce7b&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=4d146a1b44df454bbbc17abada36747a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:29.890729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 101/201] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:29.907995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:29.908713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:29.909015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:29.950767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:29.952146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:30.060993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 101/202] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:30.088656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4513-eeef-47c0-8b12-1066c62e8178 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:30.339093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4513-eeef-47c0-8b12-1066c62e8178 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:30.359002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:30.389590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:30.390397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:30.390680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:30.390954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:30.408215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:30.421799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:30.422711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:30.423469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 102/203] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:30.428978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:30.657035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:30.677086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0affa83-b764-4f4a-8c07-7b0d67b18b63 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:30.758294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:30.758972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:30.759257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:30.759531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:30.760309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:30.760622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:30.762095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:30.785815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:30.791938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:30.982931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bd6fd-8214-40a7-9e7f-af38d1b74ac7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:31.030516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:31.031392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.031674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.031946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] PATH_INFO: `/v3/users/599fadf17127442e90896add7cae1ca4` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.033145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] RBAC: Authorizing `identity:get_user(user_id=599fadf17127442e90896add7cae1ca4)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:31.034639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:31.036249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:30 2022] GET /identity/v3/users/599fadf17127442e90896add7cae1ca4 => generated 289 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:31.042004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:31.191659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c532f-7605-4aca-8851-b06bc40ed6f5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:31.246043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:31.246705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.246991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.247922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.282942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:31.332310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:31.333818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:31.446601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 103/206] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:31.452235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:31.509711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:31.565929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c3c6-b166-4b64-9072-f2349e5c1bdc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:31.600617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:31.601773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.602232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.602650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.604125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:31.605850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcc474-594e-490a-ad3e-d8724d3250c0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:31.608724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:31 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:31.614729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:31.688260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:31.747375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0620aba-fe63-4c67-8a89-5871293b557a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:31.797466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:31.798190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.798482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.798819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.799666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, scope.project.id=223989f934384b859e00e27e484a6892, user.id=599fadf17127442e90896add7cae1ca4) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:31.799983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:31.801462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ec25a-fd31-4d0f-8775-5d85df756f6f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:31.808195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:31 2022] GET /identity/v3/role_assignments?user.id=599fadf17127442e90896add7cae1ca4&scope.project.id=223989f934384b859e00e27e484a6892&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:31.866845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35482-e85f-4856-b4ab-be47b0882e88 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.867711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35482-e85f-4856-b4ab-be47b0882e88 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.868144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d35482-e85f-4856-b4ab-be47b0882e88 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.870126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:31.876377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.876671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.876961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.895988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:31.901800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:31.951332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4cd7-a431-41af-8ac3-3fd884b9a4c4 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.951640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4cd7-a431-41af-8ac3-3fd884b9a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.951911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd4cd7-a431-41af-8ac3-3fd884b9a4c4 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.953527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 106/210] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:31.959379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:31.960087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:31.960562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:31.979298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:31.980868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:32.026521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 105/211] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:32.082449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.083954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 107/212] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:32.121697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.162158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.187075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.231423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebd8fb-a5cd-430e-99da-cfb3c6fa9d70 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.272083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb70be-6d19-42ad-b15a-e04abae1c39e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.287300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.288497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.288908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.289292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.291602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:32.293742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:32.297841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 106/213] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:32.309374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.321950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.322998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.323936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.324359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.338539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:32.347649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:32.348537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:32.349326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:32.353823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 108/214] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:32.362364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.437645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57d33-5689-41dd-9c33-9e03f561d5e7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.456151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.477963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.478649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.478940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.479219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.480397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:32.483154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73c253-9bb7-4723-9b5e-8b91ef728997 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:32.484088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.490218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b0f43-51f1-43f7-b478-d633827d0944 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.527361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.528240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.529332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.529332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.530394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:32.530780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:32.532300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:32.543861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 109/215] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:32.550703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.598178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 107/216] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:32 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:32.600858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e76301-d100-4d81-aee5-8354990be7b2 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.601432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e76301-d100-4d81-aee5-8354990be7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.602199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e76301-d100-4d81-aee5-8354990be7b2 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.604272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 108/217] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:32.612016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c845-1b42-4a96-bdd0-cd05405f37af None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.612744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c845-1b42-4a96-bdd0-cd05405f37af None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.613231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9c845-1b42-4a96-bdd0-cd05405f37af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.633958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cfd9c845-1b42-4a96-bdd0-cd05405f37af None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:32.635664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cfd9c845-1b42-4a96-bdd0-cd05405f37af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:32.675810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd89-5a0c-4edb-b95e-b262dcd04b1a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.718077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.725369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.725369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.725369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] PATH_INFO: `/v3/users/fc7f3d0bc1fd499c919d4d5f35544953` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.725693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 109/218] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:32.732314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ef13-77b8-4883-8b6a-1fb9bdb21ece None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.732661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ef13-77b8-4883-8b6a-1fb9bdb21ece None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.732890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ef13-77b8-4883-8b6a-1fb9bdb21ece None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.734334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 110/219] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:32.740021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.740380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.740730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.741900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] RBAC: Authorizing `identity:get_user(user_id=fc7f3d0bc1fd499c919d4d5f35544953)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:32.743645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:32.745608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 110/220] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/users/fc7f3d0bc1fd499c919d4d5f35544953 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:32.746440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.759703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:32.760944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:32.768183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.840172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 111/221] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:32.840959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:32.864680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014fc17-9f27-46d3-8c14-b9841467c06f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:32.972436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:32.977666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:32.978476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:32.978818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:32.979101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:32.992446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:33.003965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:33.004885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:33.005707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:33.006297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 111/222] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:33.008118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.063886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259209bb-c9dc-46b6-8036-058ae64e00fc None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.108147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.108807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.109236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.109557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.113859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.132314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca8b-f3f3-4412-b374-5317b4daba15 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.176610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.177915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.178208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.178671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.192128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:33.195008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.196507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:33.203355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:33.204239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:33.205014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 112/223] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:33.210050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.233774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.315923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 112/224] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:33.315923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.339952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4675c-7775-41ff-a066-b5c4f94f706d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.384499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.385958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.386705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.387187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.388717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:33.389560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.391174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.394631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.402986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 113/225] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.405220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.436594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2c014-a934-424a-b5ca-ba1445c122b4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.481416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.485289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.486434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.486883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.487686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.491231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:33.491231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.491231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.503711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 113/226] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.508807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.533164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1093833-b4f7-4e84-bf79-c27a31140e01 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.572013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.573247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.573753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.574174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.575637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.577397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.579587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 114/227] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.580577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.630845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009aaf7-41a7-4e05-b48b-2fa7be2960f2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.648729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.667724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.679397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 114/228] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.679809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.686020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef455fd-d658-4c2c-b873-c6f3b92e9ea1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.720736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.721971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.722458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.722952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.724334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.726047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.732594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 115/229] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.733698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.741688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:33.784705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0ea19-800f-4f94-ac1c-f3a1760a49df None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.822582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.823229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.823518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.823800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.824633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=fc7f3d0bc1fd499c919d4d5f35544953) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:33.824951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.828081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.840058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dde091-1241-49fb-957c-f0f5a1c19162 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:33.841899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/role_assignments?user.id=fc7f3d0bc1fd499c919d4d5f35544953&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.843035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.880383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:33.881717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:33.882240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:33.882711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:33.884124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:33.886037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:33.888047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:33.895947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:33.904803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:34.028459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24592ee7-a46d-4617-b5f2-3f9b3b9a5bc1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:34.061372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80820a-6e92-4e17-96f3-e7623750f256 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:34.065758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:34.077923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:34.079112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:34.079222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:34.079971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:34.080721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:34.082345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:34.086678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 116/232] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:33 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:34.094877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:34.122578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:34.123578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:34.123862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:34.124143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/e57981725c734fe3b61ccf7487f2ce7b/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:34.126217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, user_id=e57981725c734fe3b61ccf7487f2ce7b, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:34.127701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acb0d8-cc68-4727-894c-9aeb163451b9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:34.208127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0ed4-5d89-4d85-aa89-8d56260c3326 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:34.232048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 117/233] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:33 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/e57981725c734fe3b61ccf7487f2ce7b/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:34.250044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:34.251503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:34.252010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:34.252494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/e57981725c734fe3b61ccf7487f2ce7b/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:34.255228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, user_id=e57981725c734fe3b61ccf7487f2ce7b, role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:34.257243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f4141-6f8b-4f30-bf19-a99af145ca8d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:34.358043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 117/234] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:34 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/e57981725c734fe3b61ccf7487f2ce7b/roles/4d146a1b44df454bbbc17abada36747a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:35.155886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c790-f0e8-41c2-ad14-b747c9380b2d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.156386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c790-f0e8-41c2-ad14-b747c9380b2d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.156459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c790-f0e8-41c2-ad14-b747c9380b2d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.158097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 118/235] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:35.163642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.164041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.164209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.186551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:35.187664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:35.263565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:35.271162 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7690-6185-482a-98d3-5f665704bb74 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.271453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7690-6185-482a-98d3-5f665704bb74 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.271861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd7690-6185-482a-98d3-5f665704bb74 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.273232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 119/236] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:35.280001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.280293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.280580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.285196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 118/237] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:35.315779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:35.316978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:35.365743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:35.390429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:35.410143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 120/238] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:35.500721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:35.503574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a5577-2800-4f7e-9fbc-e2478431f706 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:35.538647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:35.539785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.540540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.541031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.552379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:35.560632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:35.561542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:35.562331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:35.564677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 119/239] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:35.570781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:35.644311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba7c7d-f91f-42e4-bee4-9aa95d57fdea None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:35.654389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:35.719167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:35.720731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.721313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.723240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8004e-b8dc-44a0-bf83-d7fe54623fa3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:35.723703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.726303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:35.727911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:35.730265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 121/240] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:35.735242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:35.803654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:35.804556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.804983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.805423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.806311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:35.806836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:35.808361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:35.829087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 120/241] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:35.842376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:35.862541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:35.898416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcdd71-e33b-4a5b-9ab7-da940434f4ed None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:35.971338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:35.972018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:35.972307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:35.972612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:35.973354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:35.974865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:36.034075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bfcb4-73c2-43b6-921a-6b574e148b3c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:36.041747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:36.071137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:36.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:36.072296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:36.072570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] PATH_INFO: `/v3/users/27d85f5bc93b454181b3f8b5fffb3d48` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:36.083860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 122/242] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:35 2022] POST /identity/v3/users => generated 293 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:06:36.088178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] RBAC: Authorizing `identity:get_user(user_id=27d85f5bc93b454181b3f8b5fffb3d48)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:36.089783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:36.093291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 121/243] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:35 2022] GET /identity/v3/users/27d85f5bc93b454181b3f8b5fffb3d48 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:36.098963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:36.260815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc71ab3-8092-4dc4-b529-12d8fe8f6cd6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:36.289494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:36.312910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:36.313603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:36.313891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:36.314172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:36.400670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:36.402217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db642a5-3cad-46b9-a6c4-ef4c0282307d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:36.524152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 123/244] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:36.530752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:36.642712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e196c-0583-44a7-bc8c-27be18a22f81 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:36.643040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e196c-0583-44a7-bc8c-27be18a22f81 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:36.643314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e196c-0583-44a7-bc8c-27be18a22f81 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:36.645990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 124/245] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:36.651595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:36.651836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:36.652090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:36.682035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cc145-7d63-4f78-824a-b84a67aa1169 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:36.689642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:36.691668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:36.709591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:36.732276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:36.733800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:36.734501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:36.735379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:36.737160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:36.739614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:36.741901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 122/246] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:36 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:36.749613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:36.803464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 125/247] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:36.810218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:36.839576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:36.972779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde79e1b-190d-4209-92e9-fd0dbdd349f0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.046035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:37.046997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.047488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.047947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.048979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=27d85f5bc93b454181b3f8b5fffb3d48) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:37.049499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:37.051159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386c045-24fc-4083-b81f-645b7eda9995 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:37.063557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 123/248] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:36 2022] GET /identity/v3/role_assignments?user.id=27d85f5bc93b454181b3f8b5fffb3d48&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:37.065951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ffed-c115-482f-8c24-7d42b7a0901d None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.069485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ffed-c115-482f-8c24-7d42b7a0901d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.070021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ffed-c115-482f-8c24-7d42b7a0901d None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.072763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 124/249] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:37.079683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.080260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.080721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.103015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:37.104584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:37.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ed867-c746-4e4e-9766-a9d3149affb9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.192271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:37.193417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.194022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.194485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.217362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:37.225977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:37.226937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:37.227716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:37.242053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 125/250] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:37.243983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:37.250518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:37.263240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:37.281754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:37.312579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:37.407235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3a901-f7c9-4f83-ac34-7497482705aa None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.459342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe74360-82e3-41ff-baba-d71f02a5d14f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.477990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:37.478665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.478947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.479222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.479973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:37.480281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:37.486913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3745eab-8562-40d0-8e23-753c04266681 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:37.503378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 126/252] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:37.506024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7cac-6071-4d38-b370-6da8900fde7a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.506314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7cac-6071-4d38-b370-6da8900fde7a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.506597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d7cac-6071-4d38-b370-6da8900fde7a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.511425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 127/253] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:37.514106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d7cac-6071-4d38-b370-6da8900fde7a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:37.534052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:37.535200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.535632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.536053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.557613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d07028be-b7af-45b1-be04-18816db5d384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:37.577142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:37.578187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:37.579119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:37.585622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 127/254] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:37.587238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.589588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.590047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.621641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:37.624733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:37.685879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:37.715752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d7cac-6071-4d38-b370-6da8900fde7a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.752009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:37.759513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:37.760626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:37.761050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:37.761497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] PATH_INFO: `/v3/users/fc7f3d0bc1fd499c919d4d5f35544953` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:37.763755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7f3d0bc1fd499c919d4d5f35544953)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:37.765480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:37.767550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 128/255] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:37 2022] GET /identity/v3/users/fc7f3d0bc1fd499c919d4d5f35544953 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:37.768455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:37.770599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 128/256] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:37.772994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:37.967533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf021da-cd8a-4632-a256-2a2ece9ce31e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:37.990131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3404b0-6a4e-49e1-93d8-d035c6419274 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.043586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.044475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.044758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.045032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.049605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.051104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.053028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 129/257] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:37 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:38.060052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.063877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.064552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.064834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.065109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.065893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:38.066200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.067584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.087829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 129/258] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:38.088676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.136177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.137666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.179420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a3d06-c18d-4b11-a529-54bfe363344d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.218019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.218894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.219182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.219455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.230700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:38.239574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:38.240534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 130/259] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:38.242411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.350121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd69240-64d5-4238-b84e-03424e9a00f5 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.376278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.387389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9400-ace8-44e9-bf02-dbe771057117 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.398039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.398931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.399265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.399544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.400771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.402273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.405333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 131/260] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:38.406142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.463858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.464759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.465045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.465329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/fc7f3d0bc1fd499c919d4d5f35544953/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.470916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, user_id=fc7f3d0bc1fd499c919d4d5f35544953, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.472348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.530161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682b15d-71ae-4d10-a860-fb4ac1831d3f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.556725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 130/261] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:38 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/fc7f3d0bc1fd499c919d4d5f35544953/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:38.557684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.566841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.567598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.567889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.568175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.568946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:38.569263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.570730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.580162 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 132/262] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:38.585888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.647545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.662111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.692490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f7878-2604-4eed-9c09-c5ca0881627f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.707434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78869994-2d93-4533-a3bc-0e07df93ac0d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:38.739513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.741684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.742223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.742648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.748058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:38.748364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:38.749009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:38.749561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] PATH_INFO: `/v3/users/e57981725c734fe3b61ccf7487f2ce7b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:38.751419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] RBAC: Authorizing `identity:get_user(user_id=e57981725c734fe3b61ccf7487f2ce7b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.752939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.759565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 133/263] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/users/e57981725c734fe3b61ccf7487f2ce7b => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:38.760726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:38.834922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:38.836775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:38.936047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:38.963815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 131/264] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:38.970148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.000244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aafc4-e14f-4b2b-9d49-402a1bf421ca None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:39.039823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:39.054816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:39.061024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.061024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.061024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.092756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f5ae3-fc89-444d-ae2a-4f541549aff4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:39.132362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:39.133534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.133983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.134648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.136715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:39.138808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:39.139428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:39.140368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:39.141820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:39.149014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.212836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:39.261622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 134/266] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:39.264946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.289705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e3704-5e8d-4a69-abd3-e4c522df5382 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:39.291992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:39.340453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:39.342129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.342737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.343166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.344753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=e57981725c734fe3b61ccf7487f2ce7b) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:39.345393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:39.347435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:39.358248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 133/267] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:39 2022] GET /identity/v3/role_assignments?user.id=e57981725c734fe3b61ccf7487f2ce7b&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:39.386735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d089-1c31-42e8-9e84-3d7ed866b8fc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:39.430969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:39.432110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.432567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.432986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.434582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:39.436380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e4d6-d3d2-41c5-b5d8-eb684fb987d9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:39.438700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 135/268] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:39 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:39.444395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.594923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee96d86-5de1-4a13-bc77-07680887e6fc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:39.613435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:39.636316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f357-1198-4be9-8fb4-486aa60c666e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.637576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f357-1198-4be9-8fb4-486aa60c666e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.638846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f357-1198-4be9-8fb4-486aa60c666e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.643363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 136/269] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:39.646852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:39.647522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.648974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.649445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.650106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=e57981725c734fe3b61ccf7487f2ce7b) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:39.650416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:39.651630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.651896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a78f48-29f4-4881-b713-88ce636e53a3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:39.652328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.653392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.663447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 134/270] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:39 2022] GET /identity/v3/role_assignments?user.id=e57981725c734fe3b61ccf7487f2ce7b&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=4d146a1b44df454bbbc17abada36747a => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:39.694607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:39.695825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:39.722330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47057a-393b-481b-9f1c-d47670ec3c9d None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.723397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47057a-393b-481b-9f1c-d47670ec3c9d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.723901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47057a-393b-481b-9f1c-d47670ec3c9d None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.725921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 135/271] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:39.732764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:39.733424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:39.733936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:39.765259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:39.766866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:39.829766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 137/272] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:39.883364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.884134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 136/273] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:39.911059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:39.934964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.040285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a7914-5e3d-4286-88f8-2a181489954a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.044651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.054452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039c8c8-d2ab-440c-a967-5a6737658152 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.100054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.100860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.101141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.101442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.102634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.104082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.113084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 137/274] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:40.114177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.119386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.120507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.120996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.122030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.139125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:40.149679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:40.150688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:40.151478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:40.156634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 138/275] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:40.162863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.258223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e65640-ab79-49a4-acb2-a929faaf8132 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.286898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.296751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.302041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.302041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.302041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.302041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.302041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6efd9f-1f60-43b8-a2c4-db5e31d2895b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.311852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9464-e598-4c19-a794-b48e4dcc519b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.347775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.348506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.348799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.349131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.349910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:40.350224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.351745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.362163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 139/276] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:40.367453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.436242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 138/277] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:06:40 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:40.436242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589456-08a5-4c32-8a30-e4bad541b31a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.436242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589456-08a5-4c32-8a30-e4bad541b31a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.436242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59589456-08a5-4c32-8a30-e4bad541b31a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.436242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 139/278] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:40.438041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.438339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.438622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.462931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:40.464845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:40.500525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77180349-3e48-4b4e-bd13-6789e73d32f7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.501567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.552550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.553991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.554374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.554728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] PATH_INFO: `/v3/users/5e00ab71e6284365b51c692bc61568c9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.569769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 140/279] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:40.575535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.579844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] RBAC: Authorizing `identity:get_user(user_id=5e00ab71e6284365b51c692bc61568c9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.581511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.583646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 140/280] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/users/5e00ab71e6284365b51c692bc61568c9 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:40.589252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.592559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.717915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49d2e8-fa2b-46fa-8b86-9bd8b37be63e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.723962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c7bd-396e-4005-ab83-a95b0f0cf6ab None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.763475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.764224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.764513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.764859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.765704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, user.id=5e00ab71e6284365b51c692bc61568c9, scope.system=all) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:40.766021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.766803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.767479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.767583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.767859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.768130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.773152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 141/281] 10.222.0.42 () {56 vars in 1308 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/role_assignments?user.id=5e00ab71e6284365b51c692bc61568c9&scope.system=all&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:40.785207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:40.793734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:40.794744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:40.795905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 141/282] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:40.801426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:40.876678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:40.924558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbb08-2f8a-4027-bfac-d16d3b439fa1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:40.977582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:40.978099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:40.978602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:40.978950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:40.979754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:40.980115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:40.981954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:40.992516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 142/283] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:40.998950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:41.026542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:41.147378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5164d49-7064-4910-afb8-dc246c9c2c2d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:41.196530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:41.197425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:41.197716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:41.197998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] PATH_INFO: `/v3/users/27d85f5bc93b454181b3f8b5fffb3d48` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:41.202072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] RBAC: Authorizing `identity:get_user(user_id=27d85f5bc93b454181b3f8b5fffb3d48)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:41.203598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:41.205580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 142/284] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:40 2022] GET /identity/v3/users/27d85f5bc93b454181b3f8b5fffb3d48 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:41.210717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:41.320596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537feec-d6e7-494a-814b-203133392c63 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:41.349655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:41.366945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:41.367927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:41.368119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:41.368399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:41.369634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:41.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1d745-ee20-44ed-8e3d-1abfd7abbb33 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:41.373112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 143/285] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:41 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:41.379077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:41.551837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:41.573188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c3cb-732a-4895-a5a7-a4d23258b778 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:41.618373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:41.619298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:41.619586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:41.619866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/27d85f5bc93b454181b3f8b5fffb3d48/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:41.626808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, user_id=27d85f5bc93b454181b3f8b5fffb3d48, role_id=af4dcdb2cb0f4bfe9db96cddc9eac04d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:41.628310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d397629-da0f-4ef5-bc02-d995b90f7140 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:41.775858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 143/286] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:06:41 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/users/27d85f5bc93b454181b3f8b5fffb3d48/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:41.842173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365607f8-9dd2-4275-a567-d01d60a303ff None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:41.843129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365607f8-9dd2-4275-a567-d01d60a303ff None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:41.843599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365607f8-9dd2-4275-a567-d01d60a303ff None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:41.845439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 144/287] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:41.857369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:41.857696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:41.857970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:41.878780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:41.880009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:41.988477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 144/288] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:42.029801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365607f8-9dd2-4275-a567-d01d60a303ff None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:42.168078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365607f8-9dd2-4275-a567-d01d60a303ff None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:42.206644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:42.207448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:42.207880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:42.208234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:42.221242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:42.229875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:42.231803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:42.232602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:42.234350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 145/289] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:42.240340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:42.410655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:42.440955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7839b6-f4af-4899-84c4-73476fea753d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:42.483706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:42.484420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:42.484714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:42.484995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:42.485783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:42.486097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:42.487490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:42.496530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 145/290] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:42.502427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:42.584295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:42.630779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720314d2-26ef-4788-bd4b-ec5a3d01a1b1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:42.666987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:42.667866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:42.668151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:42.669186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] PATH_INFO: `/v3/users/fc7f3d0bc1fd499c919d4d5f35544953` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:42.670562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7f3d0bc1fd499c919d4d5f35544953)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:42.672043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:42.673897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 146/291] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:42 2022] GET /identity/v3/users/fc7f3d0bc1fd499c919d4d5f35544953 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:42.679244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:42.857746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5868a52-c96b-4d77-94a6-241b7026796c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:42.905763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:42.906434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:42.906725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:42.907004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:42.916778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:42.983335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:42.985280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:43.093768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 146/292] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:43.099086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:43.245737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8ce35-adc8-4846-9db4-a91c29c74e85 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:43.280540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:43.300714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:43.301611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.302180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.302459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.303658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:43.305127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eec40a-b76c-4f8e-b097-c508b7eafe55 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:43.307150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 147/293] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:43 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:43.315007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:43.389159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:43.477955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23b4b7-c285-4d69-a4bf-c6e524de7615 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:43.521429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:43.522047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.522343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.522626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.523507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=fc7f3d0bc1fd499c919d4d5f35544953) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:43.523828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:43.525401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ccb0-4364-42fd-8066-acdfc28611d0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:43.536384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 147/294] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:43 2022] GET /identity/v3/role_assignments?user.id=fc7f3d0bc1fd499c919d4d5f35544953&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:43.636518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9224-d12a-4e3f-8d73-875f19bba4af None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.636867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9224-d12a-4e3f-8d73-875f19bba4af None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.637800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9224-d12a-4e3f-8d73-875f19bba4af None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.639540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 148/295] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:43.646106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127cb0-11fd-4c39-adca-16d80b27e489 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.646951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127cb0-11fd-4c39-adca-16d80b27e489 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.647729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78127cb0-11fd-4c39-adca-16d80b27e489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.667078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-78127cb0-11fd-4c39-adca-16d80b27e489 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:43.669225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-78127cb0-11fd-4c39-adca-16d80b27e489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:43.774734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 148/296] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:43.798733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc9224-d12a-4e3f-8d73-875f19bba4af None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:43.852899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d082d-9ac4-4648-a591-21af35af97ea None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.853295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d082d-9ac4-4648-a591-21af35af97ea None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.853509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d082d-9ac4-4648-a591-21af35af97ea None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.855123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 149/297] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:43.856364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cddde-bfce-41d0-bcc4-e9dc85acf63c None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.856738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cddde-bfce-41d0-bcc4-e9dc85acf63c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.857036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cddde-bfce-41d0-bcc4-e9dc85acf63c None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.858454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 150/298] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:43.863555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b42cf0-10cb-43dd-9fdc-3b7b62ec78e9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.863844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b42cf0-10cb-43dd-9fdc-3b7b62ec78e9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.864130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b42cf0-10cb-43dd-9fdc-3b7b62ec78e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.885916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-74b42cf0-10cb-43dd-9fdc-3b7b62ec78e9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:43.888318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-74b42cf0-10cb-43dd-9fdc-3b7b62ec78e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:43.923126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:43.933611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc9224-d12a-4e3f-8d73-875f19bba4af None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:43.972697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:43.973522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.973803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.974080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.975213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:43.981652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 151/299] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:43.981817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:43.981894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:43.981894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:43.981894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:43.983583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 149/300] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:43.984780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.001504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:44.003172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:44.055929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.081381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 152/301] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:44.081381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.100859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13e73a-7ebf-4620-bfaa-e17036634562 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.139878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.140551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.140838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.141120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] PATH_INFO: `/v3/groups` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.141857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.143318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.187986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.197140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101229a-7421-4d1d-ace7-999569bb34db None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.225614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 150/302] 10.222.0.42 () {60 vars in 1134 bytes} [Fri May 27 16:06:43 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:06:44.226599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.247506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.248972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.248972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.249250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.266064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:44.276625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:44.278353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:44.279150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 153/303] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:44.290935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.369604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4544c3c-5081-47d6-aa1b-d4c495c4c531 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.390676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.411727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.412509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.412864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.413208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.427272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9adb3a-09ad-474c-86a4-9bdf0f5ddb39 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.430130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:44.440830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:44.442046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:44.442819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:44.448732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.477131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.478138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.478425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.478704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.479454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:44.479765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.481220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.490578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 154/305] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:44.496642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.514137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.569262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8838bc6-dc04-47d7-9149-b72ba7e4068c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.605723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.606387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa9050-d371-4aed-9c8c-b4fc4528e407 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.606528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.606716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.606954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.607707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:44.608020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.609438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab90279-9571-456a-9553-2079ad5f9bac None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.619106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 152/306] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:44.620499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d27f3-0d15-419b-b775-5f7a0b037f00 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.620739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d27f3-0d15-419b-b775-5f7a0b037f00 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.621023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d27f3-0d15-419b-b775-5f7a0b037f00 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.622820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 153/307] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:44.627554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d27f3-0d15-419b-b775-5f7a0b037f00 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.649100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.650052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.650486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.650778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] PATH_INFO: `/v3/users/5e00ab71e6284365b51c692bc61568c9` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.651989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=5e00ab71e6284365b51c692bc61568c9)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.654726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a59555-82d1-474f-a0fd-4dc20e57b5b4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.656407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 155/308] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/users/5e00ab71e6284365b51c692bc61568c9 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:44.656407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.656407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.656407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.676090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:44.677299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:44.681301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.745663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d27f3-0d15-419b-b775-5f7a0b037f00 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.757362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 156/309] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:44.757362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.772856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.798513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.799397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.799685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.799967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] PATH_INFO: `/v3/users/0d7a38ee71144c759dd78e62c9043fc9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.813932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] RBAC: Authorizing `identity:get_user(user_id=0d7a38ee71144c759dd78e62c9043fc9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.815430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.817308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 154/310] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/users/0d7a38ee71144c759dd78e62c9043fc9 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:44.823823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:44.875975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1110f-6e04-4ebf-9514-63cdfce74291 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.912865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.913757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.914049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.914329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] PATH_INFO: `/v3/system/users/5e00ab71e6284365b51c692bc61568c9/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.916068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e00ab71e6284365b51c692bc61568c9, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:44.917482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:44.927301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:44.932149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7a3dd-1ff1-4672-918d-05ca292c052f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:44.965144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:44.972317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:44.972317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:44.972317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:44.976410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:44.984604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:44.985700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 155/311] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:44.991148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:45.038938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 157/312] 10.222.0.42 () {58 vars in 1334 bytes} [Fri May 27 16:06:44 2022] PUT /identity/v3/system/users/5e00ab71e6284365b51c692bc61568c9/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:45.040432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:45.123037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3c82-0c03-4d85-a3f7-451c560b7d0a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:45.141256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:45.166352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a67e-d083-4921-9c3d-55f344eeb600 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:45.174765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:45.175770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:45.176229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:45.176644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:45.178378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, user.id=0d7a38ee71144c759dd78e62c9043fc9, scope.system=all) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:45.178874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:45.180925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:45.191453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 156/313] 10.222.0.42 () {56 vars in 1308 bytes} [Fri May 27 16:06:44 2022] GET /identity/v3/role_assignments?user.id=0d7a38ee71144c759dd78e62c9043fc9&scope.system=all&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:45.219340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:45.220515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:45.221001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:45.221537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:45.222540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:45.223063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:45.224756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:45.236357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 158/314] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:45.242208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:45.273297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:45.364676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e993e2-1352-4a81-a416-ea9d9295528a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:45.407522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:45.408727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:45.409213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:45.409795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] PATH_INFO: `/v3/users/27d85f5bc93b454181b3f8b5fffb3d48` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:45.411312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] RBAC: Authorizing `identity:get_user(user_id=27d85f5bc93b454181b3f8b5fffb3d48)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:45.413039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:45.415281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 157/315] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:45 2022] GET /identity/v3/users/27d85f5bc93b454181b3f8b5fffb3d48 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:45.422611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:45.543528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f649615-6f38-4815-be63-18bdba349c74 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:45.585935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:45.586621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:45.586983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:45.587267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:45.596639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:45.661544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:45.663048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:45.770825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 159/316] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:45.776561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:45.906038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:45.915802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41f724-a0aa-4230-8efc-b1adc4b62f1b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:45.952044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:45.953069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:45.953483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:45.953604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:45.954862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:45.956343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a43d4-aa63-4eb6-a398-339a93696737 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:45.958100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 158/317] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:45 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:45.964005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:46.011070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:46.092335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaa87b-4a8c-4c04-873c-ccfdf84512dd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:46.129025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:46.129780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:46.130113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:46.130394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:46.131217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, scope.project.id=4cdae7099bdb4ddaa682931bc030222f, user.id=27d85f5bc93b454181b3f8b5fffb3d48) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:46.131529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:46.132988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d0391-a405-4b51-80b0-3e9ce1edad49 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:46.139928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 160/318] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:06:45 2022] GET /identity/v3/role_assignments?user.id=27d85f5bc93b454181b3f8b5fffb3d48&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:46.704825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9662f-931a-4717-967d-fcba69483690 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:46.705310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9662f-931a-4717-967d-fcba69483690 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:46.705608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9662f-931a-4717-967d-fcba69483690 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:46.706975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 159/319] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:46.712183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:46.712616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:46.712894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:46.730551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:46.731864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:46.801961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 161/320] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:46.823608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9662f-931a-4717-967d-fcba69483690 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:46.977979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9662f-931a-4717-967d-fcba69483690 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:47.011611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:47.013250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.013562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.013835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.015007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:47.016449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ae2a0-a4b5-422e-ac03-195a2e186931 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:47.018282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 160/321] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:47.023587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:47.054374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:47.125019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fce5ea-ae0a-4cf5-a590-66759bed07c0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:47.159144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:47.159804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.160085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.160359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] PATH_INFO: `/v3/groups` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.161116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:47.162627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75eb86-2e4a-433b-a13b-63a59921717b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:47.286301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 162/322] 10.222.0.42 () {60 vars in 1134 bytes} [Fri May 27 16:06:47 2022] POST /identity/v3/groups => generated 226 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:06:47.437165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc412-d35d-4b3f-a9fb-640dca00753d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.437606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc412-d35d-4b3f-a9fb-640dca00753d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.437768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc412-d35d-4b3f-a9fb-640dca00753d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.439196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 161/323] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:47.444714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.445015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.445306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.462908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:47.464132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:47.509075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973299d-2ad5-4db7-8b64-4b26cda3a086 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.509480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973299d-2ad5-4db7-8b64-4b26cda3a086 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.509737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973299d-2ad5-4db7-8b64-4b26cda3a086 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.511258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 162/324] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:47.516387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.516687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.516971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.534622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:47.535837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:47.546529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 163/325] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:47.558598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:47.571462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:47.603520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:47.614028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 163/326] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:47.636099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:47.709834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912c4c2-3487-45f6-a6a4-8d027107df87 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:47.758767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:47.759581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.760126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.760411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.768010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff5ca7-3e27-43cc-b9cc-72ba80d387d7 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:47.774943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:47.785695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:47.786599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:47.787362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:47.788573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 164/327] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:47.794030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:47.822577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:47.822577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:47.822577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:47.822577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:47.834224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:47.844432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:47.846243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:47.848289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 164/328] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:06:47.853723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:47.879721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:47.905635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:47.935334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6886d75-1c9b-44f6-ae39-2269a9befe2b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.004099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.004910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.005124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.005446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.006236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:48.006684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.007959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.019639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce4410-d22b-44b4-98a9-d8430572f2c7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.022017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 165/329] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:48.026972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:48.064376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.065061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.065368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.065665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.066419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:48.066726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.068235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.079248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 165/330] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:48.085459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:48.171181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdc132-1fc3-4c78-8be3-9e613bb458c7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.190497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:48.221963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c47a5-c9fe-467e-9a6d-f68fefe0e2aa None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.226148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.226891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.227171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.232160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] PATH_INFO: `/v3/users/5e00ab71e6284365b51c692bc61568c9` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.233201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] RBAC: Authorizing `identity:get_user(user_id=5e00ab71e6284365b51c692bc61568c9)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.234720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.236547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 166/331] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:48 2022] GET /identity/v3/users/5e00ab71e6284365b51c692bc61568c9 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:48.241828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:48.265207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:48.287966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.288846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.289127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.290416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] PATH_INFO: `/v3/users/0d7a38ee71144c759dd78e62c9043fc9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.292255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7a38ee71144c759dd78e62c9043fc9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.294337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.296222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 166/332] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:48 2022] GET /identity/v3/users/0d7a38ee71144c759dd78e62c9043fc9 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:48.301903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:48.443416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfed4dc-86b8-4e9c-a413-5bb7b5155f62 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.452324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b477db-d4d8-45e6-94d3-24ef60b1b762 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:48.484877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.485953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.486350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.486736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] PATH_INFO: `/v3/system/users/0d7a38ee71144c759dd78e62c9043fc9/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.488449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0d7a38ee71144c759dd78e62c9043fc9, role_id=af4dcdb2cb0f4bfe9db96cddc9eac04d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.490116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646239a5-d0f2-4f3c-964a-1459b5d85b14 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.520738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:48.521436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:48.521776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:48.522064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:48.522898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] RBAC: Adding query filter params (role.id=87496757e0464ade96d2c6d9ef7ca8ec, user.id=5e00ab71e6284365b51c692bc61568c9, scope.system=all) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:48.523220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:48.524828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2c82c-3dae-4a37-9ff1-740785550f50 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:48.538834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 167/333] 10.222.0.42 () {56 vars in 1308 bytes} [Fri May 27 16:06:48 2022] GET /identity/v3/role_assignments?user.id=5e00ab71e6284365b51c692bc61568c9&scope.system=all&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 510 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:48.555143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 167/334] 10.222.0.42 () {58 vars in 1334 bytes} [Fri May 27 16:06:48 2022] PUT /identity/v3/system/users/0d7a38ee71144c759dd78e62c9043fc9/roles/af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:50.390932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bbe75-a06d-4e3c-bc49-9faf22dee03e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.391744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bbe75-a06d-4e3c-bc49-9faf22dee03e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.391744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bbe75-a06d-4e3c-bc49-9faf22dee03e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.393230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 168/335] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:50.399761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.401180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.402080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.432238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:50.433906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:50.437454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:50.526946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 168/336] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:50.574361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bbe75-a06d-4e3c-bc49-9faf22dee03e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:50.671974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:50.714903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6bbe75-a06d-4e3c-bc49-9faf22dee03e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:50.760888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:50.762331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.762732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.763039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.779336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-7a785941-72da-4003-a7f1-478d2288e0cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:50.787513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:50.788445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:50.790109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 169/337] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:50.795170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:50.882777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2353-2f0a-41fe-8921-91af29a8d438 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.883108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2353-2f0a-41fe-8921-91af29a8d438 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.883348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e2353-2f0a-41fe-8921-91af29a8d438 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.884850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 170/338] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:50.890415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.890712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.890989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.923892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:50.925041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:50.925410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2739091-f3e9-4fe6-9ed2-5ca70429a1b8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:50.969037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:50.969756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.970042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.970316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.971193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:50.971502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:50.972892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966fb3d-d172-48a4-a1f5-6394e22795b1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:50.993612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 169/339] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:50.994934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b6ce7-9e15-42d8-b5b8-d4b9e64ca594 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:50.995209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b6ce7-9e15-42d8-b5b8-d4b9e64ca594 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:50.995493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b6ce7-9e15-42d8-b5b8-d4b9e64ca594 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:50.997989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 170/340] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:51.004572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.004572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.004572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.010517 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.024306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:51.026491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:51.039945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 171/341] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:51.040630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.112614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.139643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 171/342] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:06:51.140822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.160282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f1ab-6e84-45ab-96f1-c20b82c96cb8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.202886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.203536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.203940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.204224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.275490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6d587-0c10-42bc-9ce0-e36645b4426d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.284795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:51.286452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:51.309022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.321373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.333568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.333568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.333568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.358728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.366784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:51.390869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:51.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 172/343] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:51.395114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.413192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 172/344] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:51.419733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.591161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77070db5-34a3-4968-a781-42797cfc031f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.594033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a8b5a-b2e6-4c6c-8f02-eaf6ac5e7776 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.644027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.644309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.644585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.650138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:51.651627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:51.654348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 173/345] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:51.655142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.666762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.667638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.668511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.668789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.680773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:51.691671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:51.693976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 173/346] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:51.696006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.700084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.793751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:51.820487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40212f3e-177d-4f41-bdb1-69801367385c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.824570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6116f-6e7d-4ff3-8840-e19ac8946189 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:51.865109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.866841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.867568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.868103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.880585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:51.882351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:51.884235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 174/347] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:51.885409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:51.894093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:51.894791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:51.895078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:51.895360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:51.896120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:51.896433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:51.897869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:51.907592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 174/348] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:51.908441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.012962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541341-d2f8-43d9-a43d-7d6f867baf23 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.045887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.062535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:52.063597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.064078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.064558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.065671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:52.066237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:52.068301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:52.078097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.081808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 175/349] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:52.083014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.099385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e6a60-89a9-4cff-b7b9-ec3d1a1df64f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.174200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:52.174801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.175093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.175376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.176218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=223989f934384b859e00e27e484a6892) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:52.176539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:52.178018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:52.184772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 175/350] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:06:51 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=223989f934384b859e00e27e484a6892&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:52.193769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.205276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da10f9-30e7-4f16-8ac6-257cab99834d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.246550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:52.247715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.248238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.248771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.304803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.331668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:52.333764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cd911-6da0-4e90-9770-aadb5d9e985b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:52.368947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3494802-a7a6-433d-a502-d073ccc5faa3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.438203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:52.438822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.439076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.439361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.473575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.483021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 176/351] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:52.486363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed2e20-ab8b-42ca-9102-8b1090b43a01 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.486363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed2e20-ab8b-42ca-9102-8b1090b43a01 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.486363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed2e20-ab8b-42ca-9102-8b1090b43a01 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.487864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 177/352] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:52.489214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c545ea-2d06-433a-8469-39264857feaf None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.493723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c545ea-2d06-433a-8469-39264857feaf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.494088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c545ea-2d06-433a-8469-39264857feaf None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.495677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 178/353] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:52.496830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c545ea-2d06-433a-8469-39264857feaf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.557613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:52.559521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d45987-a671-4aae-9a10-894ef3a8aae2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:52.599360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.627662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c545ea-2d06-433a-8469-39264857feaf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.669381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:52.670602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.671113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.671598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.673064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:52.674750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea321fef-8651-4070-b0b6-3fccae444a9b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:52.676681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 179/354] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:52 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:52.677914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 176/355] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:52.678500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.679045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.679212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:52.679560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:52.679847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.680069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:52.699289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:52.700440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:52.700952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:52.702589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:52.776338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.804939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 177/356] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:52.805929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 180/357] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:52.806510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.806770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:52.916232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:52.959312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb7a36-fd0f-4414-b960-5aff56782660 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:52.971926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32eae4-52ad-41ac-8641-c3140537fadb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.010116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ca851-1662-4b88-901e-15091438414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.010991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ca851-1662-4b88-901e-15091438414a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.011412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ca851-1662-4b88-901e-15091438414a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.012281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ca851-1662-4b88-901e-15091438414a None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.012924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999c226-1c88-4909-876d-614805868550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.013871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c226-1c88-4909-876d-614805868550 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.014145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ca851-1662-4b88-901e-15091438414a None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.014318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c226-1c88-4909-876d-614805868550 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.014466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c226-1c88-4909-876d-614805868550 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.015589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999c226-1c88-4909-876d-614805868550 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.015777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ca851-1662-4b88-901e-15091438414a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.017019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999c226-1c88-4909-876d-614805868550 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.017672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 181/358] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:52 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.018756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 178/359] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:52 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.018972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf5ee-1f24-43d8-8876-cb41045591df None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.019273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf5ee-1f24-43d8-8876-cb41045591df None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.019554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf5ee-1f24-43d8-8876-cb41045591df None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.019718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999c226-1c88-4909-876d-614805868550 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.025575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 182/360] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:53.027663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbf5ee-1f24-43d8-8876-cb41045591df None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.103936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.174347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999c226-1c88-4909-876d-614805868550 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.181282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbf5ee-1f24-43d8-8876-cb41045591df None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.231269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.232097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.232379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.232660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.233303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.234219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.234557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.234912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.244790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:53.249206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:53.254594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:53.255820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 183/361] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:53.256867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:53.259355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:53.260485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 179/362] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:53.263035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.267953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.415163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91aa16-b1b5-4828-a45f-e9d9978f7d18 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.425737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f087a52-9155-4275-9756-36dbb0c9e7db None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.447130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.461707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.462867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.463287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.463617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.465115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=4cdae7099bdb4ddaa682931bc030222f) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:53.466081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.467637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a2074-dc8f-44c9-8f75-48802b25a88a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.471710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.472573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.472857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.473135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.473864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 184/363] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.477520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.477520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.477754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527615a-ed23-470e-bbae-262666012403 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.477754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527615a-ed23-470e-bbae-262666012403 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.477998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 180/364] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.478089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527615a-ed23-470e-bbae-262666012403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.481638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.513835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e527615a-ed23-470e-bbae-262666012403 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:53.515522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e527615a-ed23-470e-bbae-262666012403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:53.619944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.628047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 185/365] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:53.629333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527615a-ed23-470e-bbae-262666012403 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.645313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b44c-1b32-4b86-a6c0-099f8fc30b92 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.686797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.687483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.687768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.688044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.688819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:53.689123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.690648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.701254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 181/366] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.701952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.748711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527615a-ed23-470e-bbae-262666012403 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.756881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.801675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.802646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.803018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.803303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.804062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:53.804371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.806628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.823389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 186/367] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.823389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.831969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00c2af-bddb-419a-8164-009aa4a66945 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:53.876743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:53.878114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:53.878594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:53.879060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:53.880150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=223989f934384b859e00e27e484a6892) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:53.880658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:53.882399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:53.890221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 182/368] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=223989f934384b859e00e27e484a6892&role.id=4d146a1b44df454bbbc17abada36747a => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:53.891414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:53.899067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:53.967024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab1973-4405-4566-b936-2979b513c17e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.006007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.006711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.006998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.007276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.009865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2877-0654-422b-9ace-bc87af661dda None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.018876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.021578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:54.030563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:54.031583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:54.032363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:54.036274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 187/369] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:54.037317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.055533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.056221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.056571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.056916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.135394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.137805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.159918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee70fe-5d84-4bce-92a1-e5debff92b1a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.183284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.202275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.206155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.206155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.206155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] PATH_INFO: `/v3/users/0d7a38ee71144c759dd78e62c9043fc9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.206155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7a38ee71144c759dd78e62c9043fc9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.207891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.210442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 188/370] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/users/0d7a38ee71144c759dd78e62c9043fc9 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:54.211507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.256142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 183/371] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:54.256335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.269485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.332227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d24ff-ffdc-446f-931d-78bb22e96065 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.368956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18753105-a30c-4f19-b556-faabc5326007 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.371957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.373102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.373622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.374090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.375032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:54.375530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.377123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.387785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 189/372] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:54.389217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.407199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.407862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.408145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.408421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.409245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] RBAC: Adding query filter params (role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d, user.id=0d7a38ee71144c759dd78e62c9043fc9, scope.system=all) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:54.409599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.411189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.425706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 184/373] 10.222.0.42 () {56 vars in 1308 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/role_assignments?user.id=0d7a38ee71144c759dd78e62c9043fc9&scope.system=all&role.id=af4dcdb2cb0f4bfe9db96cddc9eac04d => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:54.426621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.448410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.508205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b044a03-5a35-44cf-a76b-21887d018298 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.523562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.549661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.551012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.551504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.552105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.553813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.555568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.557666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 190/374] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:54.565948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.588146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158822c-dba0-4267-b7e1-b489b657af96 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.638135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.638830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.639113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.639454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.760747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06efc9c-f224-4b8e-885f-13dde75498e1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.784783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:54.802584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.804090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ed526-13cf-472d-8053-e0662fc416ee None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.804508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.805427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.805773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.806055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.807400 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.808958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.811187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 191/375] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:54.817193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:54.938832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7672e-2247-4461-98e9-8b34327acba6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:54.977413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:54.978087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:54.978371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:54.978650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:54.979618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=4cdae7099bdb4ddaa682931bc030222f) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:54.981106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:54.982128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:54.992118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 192/376] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=4d146a1b44df454bbbc17abada36747a => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:55.006907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:55.069617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 185/377] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:06:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:06:55.076284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.094716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a440-e2cf-4636-bc4e-983dce19c249 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.095015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a440-e2cf-4636-bc4e-983dce19c249 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.095283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a440-e2cf-4636-bc4e-983dce19c249 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.096920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 186/378] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:55.102201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.102492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.102771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.150441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:55.151707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:55.212797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:55.291680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee1efb-7c17-4024-bcfb-91f9121dbd5c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:55.340840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0224a-e167-431a-9288-f86790348262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:55.341614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0224a-e167-431a-9288-f86790348262 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.341905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0224a-e167-431a-9288-f86790348262 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.342186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0224a-e167-431a-9288-f86790348262 None admin] PATH_INFO: `/v3/projects/b89eb764079a4814b557772bebc9a51d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.343030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 187/379] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 16:06:55.350186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0224a-e167-431a-9288-f86790348262 None admin] RBAC: Authorizing `identity:get_project(project_id=b89eb764079a4814b557772bebc9a51d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:55.351678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0224a-e167-431a-9288-f86790348262 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:55.353653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 193/380] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/projects/b89eb764079a4814b557772bebc9a51d => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:55.359527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.374644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0224a-e167-431a-9288-f86790348262 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.487112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:55.499663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8c52-b02b-4a0b-9fac-69cef35b9bc5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:55.531223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0224a-e167-431a-9288-f86790348262 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:55.563884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:55.576442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:55.577307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.577877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.578153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] PATH_INFO: `/v3/groups/fb6bb8c259094dbe9504d834174c89a5` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.591240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] RBAC: Authorizing `identity:get_group(group_id=fb6bb8c259094dbe9504d834174c89a5)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:55.592715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:55.597239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 188/381] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/groups/fb6bb8c259094dbe9504d834174c89a5 => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:55.600769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:55.601396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.601694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.601973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.604173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.629960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:55.639202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:55.640159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:55.640935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 194/382] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:55.647065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.799755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b825ce-0437-4ce2-b279-1383e8bb7077 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:55.809269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689195-0778-4c95-8238-6af446b55a9c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:55.840182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:55.841198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.841635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.842110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.843365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] RBAC: Adding query filter params (group.id=fb6bb8c259094dbe9504d834174c89a5, role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=b89eb764079a4814b557772bebc9a51d) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:55.843945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:55.845720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:55.850669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:55.851470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:55.851761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:55.852040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:55.852792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:55.853102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:55.853960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 189/383] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/role_assignments?group.id=fb6bb8c259094dbe9504d834174c89a5&scope.project.id=b89eb764079a4814b557772bebc9a51d&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:55.856768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:55.872198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 195/384] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:55.877924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:55.924664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:56.029662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920915b9-e562-4d2d-a3cc-e8d7e12da1fa None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:56.078854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:56.080166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:56.080657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:56.081143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:56.082682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:56.084485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:56.087484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 190/385] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:55 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:56.092290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:56.171691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:56.254837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9a8b-17a9-4988-924f-ed0918fb0a54 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:56.301461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:56.302379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:56.302670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:56.302946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:56.304151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:56.305641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81901a62-f1a5-4287-8268-ae818bbe2c61 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:56.307656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 196/386] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:56 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:56.313111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:56.440139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:56.450861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d6dbc-6f48-4edb-a6b3-9b8109a9fd7f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:56.492753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:56.493667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:56.493951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:56.494231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/groups/979720fe7040436986ca1d48c85cd354/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:56.496228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, group_id=979720fe7040436986ca1d48c85cd354, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:56.497744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a107f4-f367-4642-a03c-8f577f954113 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:56.582752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 191/387] 10.222.0.42 () {58 vars in 1442 bytes} [Fri May 27 16:06:56 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/groups/979720fe7040436986ca1d48c85cd354/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:56.638204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead718dd-e178-42a3-a51c-53d95a1371dd None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:56.638520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead718dd-e178-42a3-a51c-53d95a1371dd None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:56.638773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead718dd-e178-42a3-a51c-53d95a1371dd None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:56.640524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 197/388] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:56.646708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:56.647051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:56.647289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:56.669717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:56.670936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:56.791058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 192/389] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:56.827459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead718dd-e178-42a3-a51c-53d95a1371dd None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:56.966511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead718dd-e178-42a3-a51c-53d95a1371dd None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:56.976456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:57.010093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.010896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.011393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.011753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.023425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-84735039-8f6c-4ff9-a671-b4f78a64078b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:57.031302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:57.032202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:06:57.033007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:57.037453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 198/390] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:57.039216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.174545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:57.194083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1ac55-3ea6-43ed-89bb-8c432aa8ef5d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:57.242155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.242843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.243131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.243407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.244159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] RBAC: Adding query filter params (name=member) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:57.244465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:57.246561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:57.260067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 193/391] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:57.266677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60df7-72e0-473a-893c-d0177b0bbbaa None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.266828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.267328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60df7-72e0-473a-893c-d0177b0bbbaa None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.267846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60df7-72e0-473a-893c-d0177b0bbbaa None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.269472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 199/392] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:57.274569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.275077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.275509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.298401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:57.300078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:57.410269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 200/393] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:57.435676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606fc89-ac92-498c-8c21-2251cc46816a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:57.441068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.496825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.498691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.499763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.500387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.503755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:57.505453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:57.513112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 194/394] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:57.514300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.538084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:57.621173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f19f25-d31b-4f38-8511-4807b4d0c875 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:57.630288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:57.673889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.674534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.674818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.675097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.693837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:57.700574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f63a8-712d-467d-aea8-f38ad6761cd4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:57.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:57.711457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:57.713234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 201/395] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:57.718239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.744270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.745808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.746414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.746887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.748274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:57.750037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:57.752493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 195/396] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:57.758057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:57.900581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f768d65-db56-47b8-9fd7-573e6638b99a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:57.949205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:57.952747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:57.953467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:57.953908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:57.954194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:57.955053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:57.955450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:57.957057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:57.967873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 202/397] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:57.974675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.000536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:58.012770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e16e7-18d5-4eba-a914-b425b4244afc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.073320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.074954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.075238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.075514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/groups/979720fe7040436986ca1d48c85cd354/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.077711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, group_id=979720fe7040436986ca1d48c85cd354, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:58.079151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894c270-a652-4a1c-a4e3-5fd0b0a256c0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:58.098716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7935e8f-581a-42cf-99bc-6b1235ca2032 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.136550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.137441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.137731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.138010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.139200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:58.140693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f6876-e705-4106-9b78-b1b28336adaa None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:58.142354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 203/398] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:57 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:58.143554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40e6e-4c74-48bf-8596-514004ea61a6 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.143839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40e6e-4c74-48bf-8596-514004ea61a6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.144124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40e6e-4c74-48bf-8596-514004ea61a6 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.145655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 204/399] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:58.149637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e40e6e-4c74-48bf-8596-514004ea61a6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.152472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 196/400] 10.222.0.42 () {58 vars in 1442 bytes} [Fri May 27 16:06:57 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/groups/979720fe7040436986ca1d48c85cd354/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:06:58.162194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.162493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.162777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.209481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:58.210618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:58.265261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:58.281852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e40e6e-4c74-48bf-8596-514004ea61a6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.325947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.327131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.327419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.327698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.329111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:58.330954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:58.337429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 205/401] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:58 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:58.339616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.433969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 197/402] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:58.457524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.461555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf6c0d-18d6-4d25-8d27-7b1171e569a1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.508648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.509822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.510112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.510398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892/groups/979720fe7040436986ca1d48c85cd354/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.514333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] RBAC: Authorizing `identity:create_grant(project_id=223989f934384b859e00e27e484a6892, group_id=979720fe7040436986ca1d48c85cd354, role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:58.516521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:58.537635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:58.577802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542c7ce-2641-48d2-a584-cb4394d16ec6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.616010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.616961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.617256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.617561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.634102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 206/403] 10.222.0.42 () {58 vars in 1442 bytes} [Fri May 27 16:06:58 2022] PUT /identity/v3/projects/223989f934384b859e00e27e484a6892/groups/979720fe7040436986ca1d48c85cd354/roles/4d146a1b44df454bbbc17abada36747a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:06:58.638792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:58.647713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:58.650223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:06:58.652780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 198/404] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:06:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:58.657822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.681717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:58.782943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89799b-740d-4fee-b444-f8d0b82acb08 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.825980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:58.826670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.826962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.827242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:58.828008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:58.828320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:58.829753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90ea1b-999d-47e6-88bf-4b5314a82ad5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:58.843250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 207/405] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:06:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:58.846428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:58.961194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d4b7-131d-4007-9c84-819c85c65d81 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:58.990439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:58.998522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17bc77-cacc-400c-b9bf-dadd58b31f8e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:58.999095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17bc77-cacc-400c-b9bf-dadd58b31f8e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:58.999095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17bc77-cacc-400c-b9bf-dadd58b31f8e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.001145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 208/406] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:06:59.001991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.006614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.006614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.006614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.006614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:59.006878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:59.008364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.008474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 199/407] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:58 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:59.013811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.014314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.021689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.034326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:59.037321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:59.118255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 209/408] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:59.160140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.174076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:59.233741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63b340-71f5-4cbd-adaa-bb29384cce7c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.284440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee859553-92db-440a-8b91-97f094bbade0 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.326749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.328328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.329155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.330521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.331028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.332639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.333308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.334832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.337301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:59.342196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:59.345583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 200/409] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:59.351091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.356743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:06:59.369163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:06:59.370214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:06:59.371109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:06:59.377770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 210/410] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:06:59.382947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.424608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:59.465994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8875-f65b-4db3-a017-a8734f4721f2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.466544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:59.503140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.503890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fc314-aaa6-4283-b49f-97d61d1766ff None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.504163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.504435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.504717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f/groups/979720fe7040436986ca1d48c85cd354/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.507176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=4cdae7099bdb4ddaa682931bc030222f, group_id=979720fe7040436986ca1d48c85cd354, role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:59.509543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc4364-0a53-4323-80de-58a031820d8a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:59.543530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.544124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.544476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.544778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.545533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:06:59.545850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:59.547247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46576755-aed7-4425-8b35-dd45f0b8025f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:59.556084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 211/411] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:06:59.557163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0a11-e9fa-4252-b9f6-3afbc8918168 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.557516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0a11-e9fa-4252-b9f6-3afbc8918168 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.557842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0a11-e9fa-4252-b9f6-3afbc8918168 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.559988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 212/412] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:06:59.562512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f0a11-e9fa-4252-b9f6-3afbc8918168 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.582399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 201/413] 10.222.0.42 () {58 vars in 1442 bytes} [Fri May 27 16:06:59 2022] PUT /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f/groups/979720fe7040436986ca1d48c85cd354/roles/4d146a1b44df454bbbc17abada36747a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:06:59.591220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882966d-02be-446e-9aac-400ec06e169f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.591522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882966d-02be-446e-9aac-400ec06e169f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.591807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882966d-02be-446e-9aac-400ec06e169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.628557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5882966d-02be-446e-9aac-400ec06e169f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:06:59.629890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5882966d-02be-446e-9aac-400ec06e169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:06:59.685075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f0a11-e9fa-4252-b9f6-3afbc8918168 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.753692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:59.767959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:06:59.770321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:06:59.770612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:06:59.771157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 202/414] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:06:59.771302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] PATH_INFO: `/v3/groups/fb6bb8c259094dbe9504d834174c89a5` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:06:59.772738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] RBAC: Authorizing `identity:get_group(group_id=fb6bb8c259094dbe9504d834174c89a5)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:06:59.774374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:06:59.776371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 213/415] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/groups/fb6bb8c259094dbe9504d834174c89a5 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:06:59.781659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882966d-02be-446e-9aac-400ec06e169f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.813998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:06:59.889745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:06:59.951382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882966d-02be-446e-9aac-400ec06e169f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:06:59.962689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246269d-c47d-45e2-8846-cdcd2e689a1f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:00.013482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:00.014351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.014637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.014919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] PATH_INFO: `/v3/projects/b89eb764079a4814b557772bebc9a51d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.019198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] RBAC: Authorizing `identity:get_project(project_id=b89eb764079a4814b557772bebc9a51d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:00.021864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:00.023589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 203/416] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/projects/b89eb764079a4814b557772bebc9a51d => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:00.029530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:00.032999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f14395-3349-476e-9055-732e098795d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:00.034409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14395-3349-476e-9055-732e098795d7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.034701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14395-3349-476e-9055-732e098795d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.034982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f14395-3349-476e-9055-732e098795d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.064624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f14395-3349-476e-9055-732e098795d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-72f14395-3349-476e-9055-732e098795d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:00.072262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:00.073144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:07:00.074462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:00.077639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 214/417] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:06:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:00.083121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f14395-3349-476e-9055-732e098795d7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:00.160856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5c3ab-874a-45b5-890c-69aa16ed7f13 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:00.190129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:00.210626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:00.216203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:00.217104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.217496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.217786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] PATH_INFO: `/v3/projects/b89eb764079a4814b557772bebc9a51d/groups/fb6bb8c259094dbe9504d834174c89a5/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.222800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] RBAC: Authorizing `identity:create_grant(project_id=b89eb764079a4814b557772bebc9a51d, group_id=fb6bb8c259094dbe9504d834174c89a5, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:00.224301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:00.260395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f14395-3349-476e-9055-732e098795d7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:00.297100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:00.297813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.298342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.298622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.299375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:00.299799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:00.301180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:00.310537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 215/418] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:07:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:00.318472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:00.338297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 204/419] 10.222.0.42 () {58 vars in 1442 bytes} [Fri May 27 16:07:00 2022] PUT /identity/v3/projects/b89eb764079a4814b557772bebc9a51d/groups/fb6bb8c259094dbe9504d834174c89a5/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:00.461190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8796fa-bb51-403d-a51b-72d867320e53 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:00.491265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:00.518146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:00.519142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.519653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.520114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.614176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:00.615963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6d410-189c-460e-a1d3-17595759bbd1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:00.727703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 216/420] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:00.734829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:00.742168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b816-46d2-4459-86e1-8ce5b0ff6f9a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.742462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b816-46d2-4459-86e1-8ce5b0ff6f9a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.742742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1b816-46d2-4459-86e1-8ce5b0ff6f9a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.744129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 217/421] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:00.749822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:00.750114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:00.750409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:00.783176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:00.786809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:00.816645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:00.923500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 218/422] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:00.925970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666df696-9d61-4ea8-a487-843e42b31e32 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:00.965754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.014709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.015644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.015936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.016216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.019840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:01.021363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:01.027999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 205/423] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:07:00 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:01.046925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.144197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.152529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfb680-2a9c-400d-9b83-aea834352f4c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.199533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.200622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.201049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.205640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.221199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6450be-27b4-4ef2-8d93-9eeaf3a96352 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.223650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:01.242167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:01.243323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 16:07:01.244333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:01.257647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 219/424] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:01.258479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.267537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.268754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.269308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.277749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.279228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:01.280790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:01.289792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 206/425] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:01.289957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.290680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.375983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bb160-8029-4eb5-bb9b-64e3bfc2acc7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.412847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.414026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.414488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.414898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.415861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] RBAC: Adding query filter params (name=member) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:01.416316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:01.417994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06961880-f5e0-421a-a056-ce05f7e8045e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:01.427194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 220/426] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:01.433983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e12b6-8a6e-46ae-9fb5-7348bdbe9cbf None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.434460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e12b6-8a6e-46ae-9fb5-7348bdbe9cbf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.434874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e12b6-8a6e-46ae-9fb5-7348bdbe9cbf None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.436471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 221/427] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:01.437309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e12b6-8a6e-46ae-9fb5-7348bdbe9cbf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.452337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a95a1-c45d-4c70-9155-0e6f5e9078ce None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.474134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.497071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.497974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.498416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.498835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.500279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=223989f934384b859e00e27e484a6892) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:01.500998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:01.503402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99b103-0f38-4fd5-a217-e2a6c56b04d0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:01.511370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 207/428] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=223989f934384b859e00e27e484a6892&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:01.517521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.517521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.517521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.530926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.546546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:01.547788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:01.593569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e12b6-8a6e-46ae-9fb5-7348bdbe9cbf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.676138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.677601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.679639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.681894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.705704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 208/429] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:01.740767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.780364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:01.781925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:01.872795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.890194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b209c9d-28dd-4fb5-8939-b5dbbd8db06a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:01.913176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:01.915392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 222/430] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:01.922255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:01.950205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:01.953392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:01.958775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:01.958775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:01.972842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:01.982440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:01.983336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:01.984099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 209/431] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:01.990926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.077333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d94950-6350-4c83-ad10-5625c82759a3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.118310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8745b-0ea2-4a4b-8bb6-b0233df71d76 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.119587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.121103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.121430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.121725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.123109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.124676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.130935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 223/432] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:02.132175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.161175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.161894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.162181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.162460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.163271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:02.163887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.165268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.181513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 210/433] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:02.183789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.186029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.276835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a064d-d97c-41fe-b33a-52151580d453 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.304727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93b51a-ef97-47f5-b145-98c02a5836d5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.331258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.331943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.332235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.332516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.333851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.335362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.340792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 224/434] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:02.344116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.353291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.356115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.356594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.357135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.446173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.449487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.474046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9cf6-aa21-4b5d-85c8-7948569238d4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.484500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.485171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.513495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.514407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.514841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.515266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.516277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=87496757e0464ade96d2c6d9ef7ca8ec, scope.project.id=4cdae7099bdb4ddaa682931bc030222f) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:02.516775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.518483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a00f8-b7ea-49be-a2d9-31d8f56932aa None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.525672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 225/435] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=87496757e0464ade96d2c6d9ef7ca8ec => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:02.564986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f464-f8b1-4804-a40d-3e1d60094d44 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.565595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f464-f8b1-4804-a40d-3e1d60094d44 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.565710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 211/436] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 16:07:02.566123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f464-f8b1-4804-a40d-3e1d60094d44 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.568242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 226/437] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:02.569420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.574612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.575240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.575724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.595270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:02.596939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:02.693376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 227/438] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:02.704398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9a286-b2a7-4be4-8311-36b2edea93b0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.712357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.759229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.761588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.762074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.762509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] PATH_INFO: `/v3/projects/223989f934384b859e00e27e484a6892` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.763908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] RBAC: Authorizing `identity:get_project(project_id=223989f934384b859e00e27e484a6892)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.765369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.768697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 212/439] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/projects/223989f934384b859e00e27e484a6892 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:02.773428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.777921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.853043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d95297-5b57-465a-9f63-ca1f9ecf499b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.855515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:02.890347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.891147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.891436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.891701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.902196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:02.903549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928521d-b19f-4d34-a0b9-064d8fb2ac57 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-61e225cb-c941-4de4-a5f1-c331038ccad7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:02.910155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:02.915174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 228/440] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:02.915960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7ca19-07b2-44e9-a04b-eb89112af342 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.916647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7ca19-07b2-44e9-a04b-eb89112af342 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.916647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7ca19-07b2-44e9-a04b-eb89112af342 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.918731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 229/441] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:02.919446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7ca19-07b2-44e9-a04b-eb89112af342 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:02.942701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:02.943566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.943856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.944140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.945317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:02.946790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38302b4-c802-4d6b-99ca-f2edb388d18b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:02.948535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 213/442] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:02.949878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:02.950223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:02.950506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:02.967133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:02.968346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:03.031154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7ca19-07b2-44e9-a04b-eb89112af342 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.032192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.036798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 214/443] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:03.037622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.069286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.069981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.070264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.070538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.071286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:03.071593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.073572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.081764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 230/444] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:03.082550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.108503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.136180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191ad31-d56c-404c-bb1c-af0783d4a40a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.171206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.171852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.172139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.172418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.173234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=223989f934384b859e00e27e484a6892) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:03.173579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.175009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.180688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 215/445] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=223989f934384b859e00e27e484a6892&role.id=4d146a1b44df454bbbc17abada36747a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:03.181478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.183481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7db72-1cd2-4560-87d0-5869bb5a6b17 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.224845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.225742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.226026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.226300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.238841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:03.247281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:03.248299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:03.249100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 231/446] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:03.254669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.262332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.307756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8f3fa-6f67-4477-aafc-db3e53a439a6 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.350693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.351740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651deae-a415-4355-9936-015013a20c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.352543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651deae-a415-4355-9936-015013a20c73 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.352763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651deae-a415-4355-9936-015013a20c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.353040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651deae-a415-4355-9936-015013a20c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.369149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168224ab-9405-4935-bb3c-8ff1d4fe61b9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.416596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.417314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.417635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.417929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.418701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:03.419018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.420740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.431785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 232/447] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:03.442707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.461746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651deae-a415-4355-9936-015013a20c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.463247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651deae-a415-4355-9936-015013a20c73 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.564671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.604621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc2e37-ac27-470b-acb4-8a5f260dec68 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.632079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 216/448] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:03.638629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651deae-a415-4355-9936-015013a20c73 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.658924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.659586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.659874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.660155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.688433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.753493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.755028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.767346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651deae-a415-4355-9936-015013a20c73 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.807059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.808046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.808369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.808694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] PATH_INFO: `/v3/projects/4cdae7099bdb4ddaa682931bc030222f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.809946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] RBAC: Authorizing `identity:get_project(project_id=4cdae7099bdb4ddaa682931bc030222f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.811425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.813112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 217/449] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/projects/4cdae7099bdb4ddaa682931bc030222f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:03.818542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.852123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.874397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 233/450] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 16:07:03.880074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.926441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085d99-baa4-4c69-9a5f-479a15622e8b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:03.926647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:03.963357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2f16-a911-4972-b774-a837351226ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:03.964917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2f16-a911-4972-b774-a837351226ef None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:03.965207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2f16-a911-4972-b774-a837351226ef None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:03.965522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d2f16-a911-4972-b774-a837351226ef None admin] PATH_INFO: `/v3/groups/979720fe7040436986ca1d48c85cd354` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:03.966742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d2f16-a911-4972-b774-a837351226ef None admin] RBAC: Authorizing `identity:get_group(group_id=979720fe7040436986ca1d48c85cd354)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:03.968205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d2f16-a911-4972-b774-a837351226ef None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:03.970046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 218/451] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/groups/979720fe7040436986ca1d48c85cd354 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:03.975707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2f16-a911-4972-b774-a837351226ef None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:03.980655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b015cf8-9969-42fd-be82-6dad16db253e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:04.016984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:04.018304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:04.018599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:04.018880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] PATH_INFO: `/v3/projects/b89eb764079a4814b557772bebc9a51d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:04.020438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] RBAC: Authorizing `identity:get_project(project_id=b89eb764079a4814b557772bebc9a51d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:04.022255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:04.023941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 234/452] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/projects/b89eb764079a4814b557772bebc9a51d => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:04.029559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:04.076876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:04.079675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d2f16-a911-4972-b774-a837351226ef None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:04.113799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:04.114457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:04.114738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:04.115020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:04.115832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] RBAC: Adding query filter params (group.id=979720fe7040436986ca1d48c85cd354, role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=4cdae7099bdb4ddaa682931bc030222f) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:04.116143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:04.117680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:04.124363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 219/453] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:07:03 2022] GET /identity/v3/role_assignments?group.id=979720fe7040436986ca1d48c85cd354&scope.project.id=4cdae7099bdb4ddaa682931bc030222f&role.id=4d146a1b44df454bbbc17abada36747a => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:04.132748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1ab2d-4aba-453b-9f7f-7e369f218c46 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:04.149502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:04.174431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:04.175310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:04.175598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:04.177966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] PATH_INFO: `/v3/groups/fb6bb8c259094dbe9504d834174c89a5` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:04.177966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] RBAC: Authorizing `identity:get_group(group_id=fb6bb8c259094dbe9504d834174c89a5)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:04.178760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37eff8-e15a-462c-abfd-3e7f5c67d9a0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:04.180516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 235/454] 10.222.0.42 () {56 vars in 1180 bytes} [Fri May 27 16:07:04 2022] GET /identity/v3/groups/fb6bb8c259094dbe9504d834174c89a5 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:04.187655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:04.299900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552691-5f57-4fdd-8e67-092018b6034b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:04.341500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:04.342415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:04.342700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:04.342983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:04.343825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] RBAC: Adding query filter params (group.id=fb6bb8c259094dbe9504d834174c89a5, role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=b89eb764079a4814b557772bebc9a51d) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:04.344142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:04.345622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf28397-a0f9-4cc0-915b-0c6f07d6c9fa None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:04.352089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 220/455] 10.222.0.42 () {56 vars in 1376 bytes} [Fri May 27 16:07:04 2022] GET /identity/v3/role_assignments?group.id=fb6bb8c259094dbe9504d834174c89a5&scope.project.id=b89eb764079a4814b557772bebc9a51d&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:07.272967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d7e4b-be02-499d-92f5-bc1007830215 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.274150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d7e4b-be02-499d-92f5-bc1007830215 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.274666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d7e4b-be02-499d-92f5-bc1007830215 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.276872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 236/456] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:07.284007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97da2d4-bdc3-4dbf-88fc-26786d11531c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.285435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97da2d4-bdc3-4dbf-88fc-26786d11531c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.286168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97da2d4-bdc3-4dbf-88fc-26786d11531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.323526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a97da2d4-bdc3-4dbf-88fc-26786d11531c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:07.326091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a97da2d4-bdc3-4dbf-88fc-26786d11531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:07.367813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:07.416292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 221/457] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:07.456803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d7e4b-be02-499d-92f5-bc1007830215 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:07.508206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f1d-72cb-4e39-b0af-8f124f93bc85 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.508862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f1d-72cb-4e39-b0af-8f124f93bc85 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.509381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f1d-72cb-4e39-b0af-8f124f93bc85 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.511926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 222/458] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:07.534197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3544e90-21b0-4265-879f-9b651208243c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.534777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3544e90-21b0-4265-879f-9b651208243c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.535245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3544e90-21b0-4265-879f-9b651208243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.566453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d3544e90-21b0-4265-879f-9b651208243c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:07.567999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d3544e90-21b0-4265-879f-9b651208243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:07.624079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d7e4b-be02-499d-92f5-bc1007830215 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:07.699126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 223/459] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:07.699759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:07.700578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.700863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.701142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.722467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:07.724291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:07.747249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3544e90-21b0-4265-879f-9b651208243c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:07.754075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:07.755465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:07.756934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:07.756934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:07.756934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:07.756934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:07.756934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 237/460] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:07.777686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:07.850848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:07.859084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:07.888456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3544e90-21b0-4265-879f-9b651208243c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:07.941449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950ed8f-d575-4136-816c-ccda74475d21 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:07.970758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:07.971807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:07.972231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:07.972643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:07.996686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:07.999845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.002438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.003325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.004891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.005314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.007300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:08.008280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.014958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d021f-6dfd-4c68-b29e-586d49b160b0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:08.026570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:08.027622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:08.028479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:08.028479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:08.028479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.028479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:08.030353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 224/461] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:08.036803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.047394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 238/462] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:08.048761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a92ef-f624-40c9-ae42-d89eb9ce2083 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.049062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a92ef-f624-40c9-ae42-d89eb9ce2083 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.049394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a92ef-f624-40c9-ae42-d89eb9ce2083 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.050742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 239/463] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:08.058533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a92ef-f624-40c9-ae42-d89eb9ce2083 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.156111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eab676-e7b9-4232-969f-94c2263a8bbe None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:08.187128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:08.196606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a92ef-f624-40c9-ae42-d89eb9ce2083 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:08.208995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.210255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.210544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.210827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.211565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:08.212263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.213889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e3e37-66f8-4baa-970b-c277e7f6563f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.227866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:08.236488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 225/464] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:08.236488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d054-4074-4310-9ed6-298228a5d88f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.236488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d054-4074-4310-9ed6-298228a5d88f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.236488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12d054-4074-4310-9ed6-298228a5d88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.257256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.258057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.258346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.258631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.259350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.259630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dc12d054-4074-4310-9ed6-298228a5d88f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:08.261391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9d497-0be1-47ee-aae0-0e35bb620446 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.262484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dc12d054-4074-4310-9ed6-298228a5d88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:08.342423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 226/465] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:08.343658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a9dc-60e3-46a5-a703-db4243c58c6f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.343916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a9dc-60e3-46a5-a703-db4243c58c6f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.344161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a9dc-60e3-46a5-a703-db4243c58c6f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.345746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 227/466] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:08.347996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4a9dc-60e3-46a5-a703-db4243c58c6f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.398216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 240/467] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:08 2022] POST /identity/v3/users => generated 246 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:07:08.399772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.400835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.400835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.429553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:08.429553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:08.502627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4a9dc-60e3-46a5-a703-db4243c58c6f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:08.532480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 241/468] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:08.533966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.543651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:08.551559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.552269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.552555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.555820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.555820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.555820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.635186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 228/469] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:08 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:08.642714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.656643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:08.700955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d76868-7f91-4693-92b4-cad94aada5ee None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:08.774227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.776674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.776674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.776873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.793053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.794103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b336c958-4eb9-4d49-b8dc-54d805d0737c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:08.822313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:08.828351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 242/470] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd868d-fd2c-4094-8fd9-7242913c62ce None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd868d-fd2c-4094-8fd9-7242913c62ce None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd868d-fd2c-4094-8fd9-7242913c62ce None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.829331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 243/471] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:08.833056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd868d-fd2c-4094-8fd9-7242913c62ce None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:08.849310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27c39e-cd8b-41b1-985a-7794aeb5af9e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:08.893130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:08.894346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.894836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.895313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.908945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:08.910858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-29b5678f-53dd-42bd-8df0-8e3fb550c81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:08.938742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:08.939595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:08.940425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:08.940425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:08.940425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:08.940425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:08.940425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 229/472] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:08.941588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:08.941908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:08.942171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:08.960396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:08.961631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:08.984853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd868d-fd2c-4094-8fd9-7242913c62ce None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.000255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:09.026948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:09.034371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.035335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.035796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.036243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.037991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:09.039164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:09.042120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:09.043347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 230/473] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:09.050843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.073155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 244/474] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:09.075919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.182826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431078a9-8b63-47e8-bfbb-7a0aeebfea52 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.194137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abcab8-fef4-4ce7-b734-549fef8e09dc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.223227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.224249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.224747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.225310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.226358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:09.226858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:09.228522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:09.238609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.239746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.240044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.240328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.241398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:09.242825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:09.244005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 231/475] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:09.245464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.284758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:09.317418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.sql.core [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 16:07:09.317418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 16:07:09.317418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [parameters: {'id': '22a64bb8521a4a23a7fb8d09b4395862', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 16:07:09.317418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80566) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application conn.query(q) May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application result.read() May 27 16:07:09.342201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return ref.to_dict() May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:07:09.343276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application next(self.gen) May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application yield resource May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application next(self.gen) May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self.session.rollback() May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self.force_reraise() May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise self.value May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application session.commit() May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 16:07:09.344271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._prepare_impl() May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self.session.flush() May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._flush(objects) May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application compat.raise_( May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application flush_context.execute() May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rec.execute(self) May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 16:07:09.345097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:07:09.346620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application conn.query(q) May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application result.read() May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:07:09.348261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [parameters: {'id': '22a64bb8521a4a23a7fb8d09b4395862', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:09.350294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 16:07:09.351812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:09.357171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 245/476] 10.222.0.42 () {60 vars in 1131 bytes} [Fri May 27 16:07:09 2022] POST /identity/v3/roles => generated 144 bytes in 282 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:09.359116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.394411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b35af-b929-438c-b4df-80dd0e241369 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.449720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.450900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.451384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.451855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.452767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:09.454600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:09.469218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:09.530628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1cfd6-f5fc-4746-843f-6f16d7825c6c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.551940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 232/477] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:09 2022] POST /identity/v3/users => generated 249 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:09.559790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.597832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.600713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.600713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.600713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.602683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:09.606216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:09.670425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:09.690420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737e5ac-3c49-4b37-b9ab-b6631adbfb79 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.727963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 246/478] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:09 2022] POST /identity/v3/users => generated 248 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:09.743979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:09.745069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:09.745372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:09.745725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:09.769280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6bbe5800-ab65-4d59-b559-e5c2d1cb750f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:09.786454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:09.787382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:09.788412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 233/479] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:09.797032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:09.957741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ad540-6995-4bef-aa8a-8c2a684a4f04 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:09.976613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:10.003317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:10.004476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:10.004969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:10.005551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:10.006842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:10.007352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:10.008848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1800-f587-4387-b969-2b74c5d14ca5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:10.020249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 247/480] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:11.544528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ff3f-031a-4080-9362-bbcdb7b6b76a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:11.545220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ff3f-031a-4080-9362-bbcdb7b6b76a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:11.545220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ff3f-031a-4080-9362-bbcdb7b6b76a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:11.546871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 234/481] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:11.552329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3e69-7b9a-4625-ab81-fee3b598b55c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:11.552973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3e69-7b9a-4625-ab81-fee3b598b55c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:11.552973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3e69-7b9a-4625-ab81-fee3b598b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:11.578425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-611c3e69-7b9a-4625-ab81-fee3b598b55c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:11.579659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-611c3e69-7b9a-4625-ab81-fee3b598b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:11.677532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 248/482] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:11.735331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2ff3f-031a-4080-9362-bbcdb7b6b76a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:11.768407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6ed9-77c1-4d23-9cf7-5d040e79f2eb None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:11.768728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6ed9-77c1-4d23-9cf7-5d040e79f2eb None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:11.768983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b6ed9-77c1-4d23-9cf7-5d040e79f2eb None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:11.770649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 249/483] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:07:11.777479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:11.777929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:11.778373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:11.800955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:11.803313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:11.919003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2ff3f-031a-4080-9362-bbcdb7b6b76a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:11.953465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 250/484] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:11.973707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:11.992176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:11.992989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:11.993275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:11.993626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.016708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:12.024788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:12.025878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:12.026801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:12.027889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 235/485] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:12.038790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.045714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.065801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.152945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483875c-0c58-44f8-b09b-b9e32bab79c4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.191794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.192607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.192904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.193188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.211036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:12.225040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:12.226018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:12.226793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:12.232256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 251/486] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:12.235532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.273205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d25dc-8ad9-4250-a262-aa23a50614d9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.333475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.334446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.334921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.335462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.336398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:12.336882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:12.338932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:12.342086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.357854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 236/487] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:12.363373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8b9e-943a-4e2a-82a4-f084c12a1c50 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.363632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.406774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.408119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.408628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.409158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.410092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:12.410488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:12.411997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:12.426570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 252/488] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:12.428014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.496313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.561993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7c747-d02d-42cb-9240-424ec7fb1f89 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.614453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32900c-358a-4ed7-8e4c-71b48253c079 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.615168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.615835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.616126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.616421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.631934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:12.633460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6b5f0f82-90fe-4303-9543-272b9e4c950d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:12.654516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:12.655439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:12.656922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:12.656922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:12.656922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:12.656922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:12.656922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 237/489] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:12.657640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd25eee-94e7-489f-b002-91992707c1bb None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.658366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd25eee-94e7-489f-b002-91992707c1bb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.658366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd25eee-94e7-489f-b002-91992707c1bb None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.667276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 238/490] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:12.668514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd25eee-94e7-489f-b002-91992707c1bb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.672005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.673228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.673757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.674345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.686998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:12.688939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-815fba10-97e2-4afd-9fec-0386437b2107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:12.708732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:12.709675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:12.710508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:12.710508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:12.710508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:12.710508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:12.720308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 253/491] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:12.722444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.722960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.723444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.743549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:12.746012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:12.748706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.804719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd25eee-94e7-489f-b002-91992707c1bb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:12.825584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:12.833166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 254/492] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:12.834612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.857777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:12.858457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:12.858749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:12.859072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:12.859862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:12.864852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:12.864852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:12.894442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 239/493] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:12.895389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:12.975966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908991c-9d58-4295-b556-f4588f4c073a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.027041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527b2fd-f82d-4969-917b-62ae26fe680f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.032764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.034360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.035269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.036092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.038222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:13.039014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.040706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.049903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.056170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 255/494] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:13.059646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.067471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.068592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.069071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.069598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.081461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:13.090575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:13.091514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:13.092481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:13.096074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 240/495] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:13.102250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.148099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.204477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6c1-f8db-462d-90d0-0134d6964992 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.225970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47893a62-f576-4e47-8a61-c49dbca97163 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.245272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.246109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.246405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.246684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.254941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.256628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.271067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.271860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.272141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.272417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:13.272876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:13.273835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 256/496] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:13.276004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.283592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.285052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d1e9a35a-a2f7-42db-b1ec-bab21657927c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:13.301863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:13.303475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 241/497] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa30f-5b0f-4d7a-b838-66b6339e0695 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.304759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa30f-5b0f-4d7a-b838-66b6339e0695 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.305683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fa30f-5b0f-4d7a-b838-66b6339e0695 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.306327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 242/498] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:13.307457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635427-2095-4df8-9822-b4aa4f080a46 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.307691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635427-2095-4df8-9822-b4aa4f080a46 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.307907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2635427-2095-4df8-9822-b4aa4f080a46 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.313234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 243/499] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:13.313897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2635427-2095-4df8-9822-b4aa4f080a46 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.368767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.416826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f1935-480d-4515-975f-be6dcd4b8dfb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.419731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.424978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2635427-2095-4df8-9822-b4aa4f080a46 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.461221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.461930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.462217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.462494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.463311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:13.463667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.465210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c67e0-8fb0-4298-bbc8-e6977ba96b22 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.472631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.473669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.473779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 257/500] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:13.478433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.478433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.479144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00afc1b-2c1f-4146-a768-c6b819b58688 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.479144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00afc1b-2c1f-4146-a768-c6b819b58688 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.479144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00afc1b-2c1f-4146-a768-c6b819b58688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.479332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:13.479332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.479625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.493407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a00afc1b-2c1f-4146-a768-c6b819b58688 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:13.495214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a00afc1b-2c1f-4146-a768-c6b819b58688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:13.519282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 244/501] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:13.520197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.570329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 258/502] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:13.571754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.572066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.572415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.592045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:13.593296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:13.643271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.644771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4b5a1-3602-4dee-bb21-c90b52a34c4d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.664433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 259/503] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:13.665481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.708002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.708584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.708865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.709138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.709998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:13.710303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.711774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.731210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 245/504] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:13.731908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.749389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.773658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc2b92-a767-4b07-ad5a-851e877aa279 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.815879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.816690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.816986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.817266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.828606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.830134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:13.846263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c25804-8fed-4d08-bf7d-de09a5ff393f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:13.850882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:13.861292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 260/505] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:13.862156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:13.893251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:13.894166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:13.895042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:13.895322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:13.895596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:13.972334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f667f-ec03-4163-966d-8cba99a74357 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:13.982588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:13.985102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.008999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.013101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.013944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.014314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.014520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.025955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:14.033791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:14.035037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 261/506] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:14.036636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.113087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 246/507] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:14.114191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.168352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.177136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0429f36-779f-407e-aa77-ae78c62dd7aa None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.216547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.217382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.217672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.217951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.229182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.230834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:14.249858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:14.250836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 262/508] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:14.252104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.267292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67b8b-0e74-4ba6-a67f-83eeef0acc9d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.283002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.305897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.310504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.310504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.310504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.381818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cebfd-f0d4-45a0-8d95-85c13b357fc4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.388684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.390224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.419225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.420133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.420590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.421054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.422474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:14.423048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.425681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.439494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 263/509] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:14.440556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.445231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.513475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 247/510] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:14.514371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.530512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.582677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630f949-fd5b-408e-900d-1b3cb9baa869 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.630995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.631677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.631965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.632470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.632997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:14.633310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.635342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.646518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 264/511] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:14.646518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.653286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bfc55-9f32-4cae-adba-624e8b19e29e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.700792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.701619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.706264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.706264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.714935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.716482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.722851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:14.736217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:14.737149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 248/512] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:14.738620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.769758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e84eb5-7420-4a64-87d0-a8f40d5f2d2c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.812285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.816713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738abfb-1889-450a-9060-845284788a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.817501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738abfb-1889-450a-9060-845284788a7a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.817883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738abfb-1889-450a-9060-845284788a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.818234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738abfb-1889-450a-9060-845284788a7a None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.819038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738abfb-1889-450a-9060-845284788a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:14.819358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738abfb-1889-450a-9060-845284788a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.820777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738abfb-1889-450a-9060-845284788a7a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.834380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 265/513] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:14.835238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738abfb-1889-450a-9060-845284788a7a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.851039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cc508-b3ea-4c96-9ada-e36133907805 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.894691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.895493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.895779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.896055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:14.904648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:14.906137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:14.922451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:14.923418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:14.924298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 249/514] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:14.924861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:14.951151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738abfb-1889-450a-9060-845284788a7a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:14.981807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:14.992571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:14.993561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:14.993963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:14.994567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.006678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.008324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:15.028399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:15.029399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:15.030342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:15.030342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:15.030342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:15.030342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:15.030342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 266/515] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:15.030613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.056956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a784f-cd0e-43c0-86eb-5c3eca4064c0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.086021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:15.095936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.102542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.102542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.102542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.108690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.110357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:15.137599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:15.138651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 250/516] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:15.139630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.149672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebc698-a4fc-4317-b790-74137a978f9f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.190525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.191681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.192206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.192713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.194228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:15.194598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.197333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.225210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 267/517] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:15.225210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.254220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59c522-4ab0-4fa7-9771-e28d3efd65e4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.271394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:15.295886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.296473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.296757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.297031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.297786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.299236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.343834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7c730-afdf-44e3-a2a4-683cbfb6bf69 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.379879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.380582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.380866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.381146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.382067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:15.382380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.383905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.429836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 268/518] 10.222.0.42 () {56 vars in 1139 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:15.430779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.437551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 251/519] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:15 2022] POST /identity/v3/users => generated 247 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:15.438477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.485907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:15.595273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:15.595758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb199d-3cba-4606-a88a-a1d93e03827f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.604728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64317d8f-93d1-483a-8c01-bf8883da09b5 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.646915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.647360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.647646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.647917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.648985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:15.648985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.651645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.661508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.662616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.663327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.663676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.664427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:15.664567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 252/520] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:15.665074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.665398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.667145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.683392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 269/521] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:15.683897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.794913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:15.802957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ceab-2127-4d4c-aecb-20791f6894e8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.840882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930852-3798-49c2-a2bd-d6b3bcaddf85 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:15.852004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.852706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.852993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.853305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.862417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:15.864183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:15.890121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:15.890995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:15.891985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:15.892965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:15.893542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:15.893983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:15.894401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:15.900322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 253/522] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:15.902091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:15.934528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.004338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.006949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.036725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed19b79-7089-4522-ac8a-5ec6e3f2542b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.083553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.084862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.085211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.094874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.096451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:16.116156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:16.117319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:16.118304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 254/523] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:16.121602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.144620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.149751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 270/524] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:16.149751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.239622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.276077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be84dad-bd31-4f75-94fb-1e55bee4074a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.295433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992037d1-08c8-4058-88be-c1fc2c3a7553 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.326928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.328325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.328831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.329315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.343266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.346068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.348583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.349254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.349590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.349872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.350773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:16.351085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.352528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.375659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 271/525] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:16.376846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:16.377379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:16.378196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:16.379021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:16.381656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 255/526] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:16.382636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.489122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.523709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02d662-c303-4329-a63e-0fd46047b560 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.526448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19e55-cf2c-4be9-8071-8781b70ff4f1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.553569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.573711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.574519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.575026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.575135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.584164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.584858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.585140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.585452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.586399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:16.586728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.589011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.589381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.590894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.610975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 272/527] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:16.611697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:16.613468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:16.614308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:16.615165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:16.615165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:16.615165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:16.615165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:16.617096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 256/528] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:16.618107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.771346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cd987-b91d-447c-9c42-636eeba976a0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.772242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cb9af-c9c7-412e-9a3e-a7e3fbdeb638 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:16.807743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:16.817499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.817499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.821880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.821880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.821880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:16.821880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.821880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.829476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:16.830201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:16.830487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:16.830989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:16.831848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:16.832165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:16.833666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:16.849668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 273/529] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:16.849668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.865970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 257/530] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:16.871162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:16.908773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:17.030072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbbf3e-238c-49b4-9450-be3658fe1533 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:17.043343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5be28-8ace-4201-924a-4723c5a4172f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:17.084303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:17.084978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:17.085264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:17.085804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:17.096942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:17.097723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:17.098215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:17.098702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:17.100060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:17.100491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:17.102035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:17.117614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 258/531] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:17.119832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:17.182927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:17.196146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:17.204995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:17.250034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9cb0-7ff4-446a-b590-975d228afb1a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:17.265269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:17.311631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:17.312242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:17.312522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:17.312796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:17.313921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=e7017c7303f94b6380b99cb0b09d7e56) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:17.314034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:17.315502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:17.332335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 259/532] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:17 2022] GET /identity/v3/role_assignments?user.id=e7017c7303f94b6380b99cb0b09d7e56&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:17.332335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:17.401929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 274/533] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:17.404632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:17.616821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e458-14d7-435a-8e49-c351770ff55f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:17.626782 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf5fe6-0d64-4e1c-855e-0d96c5faa927 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:17.642602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:17.702104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:17.703329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:17.703808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:17.704276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:17.707367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:17.708725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:17.709183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:17.709688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:17.730421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:17.737956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:17.740735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:17.751824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:17.754572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:17.804384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:17.807091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:17.808936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 260/534] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:17.817459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:17.819600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:17.821435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:17.821435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:17.821435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:17.821435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:17.830167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 275/535] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:17.839037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:17.840082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:17.986047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9014f-ecb1-4fe5-86db-b50304f109cd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.029073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.029777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.030065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.030346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.031153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:18.031468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.033184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099f97c-9ceb-48ba-8294-08b0c8dbaac6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.057652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fddabc-fd79-4f68-a77c-8141390c41a9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.058108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 261/536] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:18.061002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde2dc-9938-441f-9b5f-06de31b1c30f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.061843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde2dc-9938-441f-9b5f-06de31b1c30f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.062571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffde2dc-9938-441f-9b5f-06de31b1c30f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.065646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 262/537] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:18.071989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde2dc-9938-441f-9b5f-06de31b1c30f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:18.097777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.098462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.098753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.099036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.100102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:18.100428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.101895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdc53-c9f1-471d-bfc9-9ffec38a7bb6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.119890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 276/538] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:18.120450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.120750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.121039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.140245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:18.141483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:18.149465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:18.203644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:18.245784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 277/539] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:18.246833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:18.305064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffde2dc-9938-441f-9b5f-06de31b1c30f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.379608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.381517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.382466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.383253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.385418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=5e62e836ab6e4f42814b3151bfe686d2) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:18.385663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2977bd-d192-430d-a7aa-d6d7099fc176 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.386590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.389874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.409548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 263/540] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/role_assignments?user.id=5e62e836ab6e4f42814b3151bfe686d2&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:18.411876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:18.427301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.428182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.428472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.428753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.437829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.439960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:18.457090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:18.458745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:18.460382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 278/541] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:18.468610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:18.519102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:18.596489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543ff0-b4a3-480b-b97b-f52937bbc24a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.646374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.647261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.647556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.648467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.649310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:18.649724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.651982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.676375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 279/542] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:18.683663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:18.807858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:18.814961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38222802-9b9e-4e68-b78f-c942db07e8f7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.819797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:18.829121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a907d-f0cc-45a8-b623-6c661a449602 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:18.898137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.899294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.899791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.905364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.905364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=2be41ec2de3948e6a015dfe35c47a0a8) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:18.905364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:18.905364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:18.933029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:18.934366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:18.934789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:18.935211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:18.936489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 280/543] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/role_assignments?user.id=2be41ec2de3948e6a015dfe35c47a0a8&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:18.960176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:18.983705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:18.984672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:18.985518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:18.990498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 264/544] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 580 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:18.997330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:19.167127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcf073-a45b-46e6-aef6-cdee2b0e5d2e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:19.229443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:19.230135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:19.230423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:19.230703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:19.231460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:19.231774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:19.233180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:19.242845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 281/545] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:19.250862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:19.394370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3fd5-aeb3-4ab8-bc78-9382440e71bd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:19.452149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:19.452889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:19.453177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:19.453484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:19.470215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:19.471754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:19.494748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:19.496279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:19.499452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:19.499452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:19.499452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:19.499452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:19.499452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 265/546] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:19.502853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:19.553509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:19.653592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b3d3b-4124-4c75-90ff-cea945b69937 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:19.702017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:19.703141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:19.703552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:19.704005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:19.705252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:19.705820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:19.708186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:19.721918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 282/547] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:19.727079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:19.737219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:19.871074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611016b7-8a68-4122-9e34-5fa92dce280a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:19.954658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:19.955873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:19.956348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:19.956804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:19.974532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:19.977081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:20.015509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:20.016492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:20.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:20.018711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 266/548] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:20.023893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:20.173791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:20.199937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb17ca-a596-4bfe-8cd7-f6bd6d227da0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:20.274774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:20.277649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.278542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.279396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.281447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:20.282391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:20.284373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d952eab-3a29-4de4-bbe5-cb72f75bdc49 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:20.320015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 283/549] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:20.326937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:20.425181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac685b4-2132-4e81-8905-716ca1219a07 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.425519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac685b4-2132-4e81-8905-716ca1219a07 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.425833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac685b4-2132-4e81-8905-716ca1219a07 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.427768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 284/550] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:07:20.434107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097cc4f-4979-46b1-9e59-c3f3ee1e9bdc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.434512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097cc4f-4979-46b1-9e59-c3f3ee1e9bdc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.434807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097cc4f-4979-46b1-9e59-c3f3ee1e9bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.445793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:20.481589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a097cc4f-4979-46b1-9e59-c3f3ee1e9bdc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:20.485965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a097cc4f-4979-46b1-9e59-c3f3ee1e9bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:20.554927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dedee2-e1c0-4a05-b416-938ba05e563f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:20.621551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 285/551] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:20.623157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b842f8-8182-4129-923f-0e7687500bfa None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.623458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b842f8-8182-4129-923f-0e7687500bfa None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.623745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b842f8-8182-4129-923f-0e7687500bfa None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.625112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 286/552] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:20.631579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:20.632331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.632619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.632899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.642067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:20.642408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:20.642691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:20.664843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:20.666379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:20.750876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:20.753675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:20.781260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 287/553] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:20.782251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:20.788273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:20.869399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:20.961192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 267/554] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:20.963675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:20.968966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d9afd-649b-484a-a4da-86b504207308 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.007408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.008622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.009106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.010255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.022097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:21.030958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:21.032522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:21.033401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:21.044323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 288/555] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:21.046355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.119753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce2efa-9754-44d3-a4b1-b4b6a57aba4b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.120827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.171744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28932aa-1b20-431b-82e6-9dd6218d81f2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.180552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.182479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.183259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.184032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.208370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:21.219179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:21.220162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:21.221040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:21.221674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.221674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.221674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.221674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.229381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 268/556] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:21.229381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.242395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.245126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.269985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:21.270529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:21.272797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 289/557] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:21.274605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.376333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c3b38-3a98-4b87-85b3-07a3c4fdefc0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.415787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60407a68-cfdd-4d35-bf8a-4bcd961f07b2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.424155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.425230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.430610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.430610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.430610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:21.430610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.430610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062494-1d87-4fc3-8a4c-9cd37f984dbe None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.442775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 269/558] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7c7c-da71-426e-b009-27da426e097f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7c7c-da71-426e-b009-27da426e097f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7c7c-da71-426e-b009-27da426e097f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 270/559] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:21.450936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7c7c-da71-426e-b009-27da426e097f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.477143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.478193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.478857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.479160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.480440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:21.480754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.482365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.505186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 290/560] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:21.506490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.585693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.605739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7c7c-da71-426e-b009-27da426e097f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.656128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.657004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603f60f-6c31-4d68-9924-ab8ef88698f1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.657530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.658035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.658689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.660484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:21.661022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.662970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc22768-3578-407f-adb8-b20f02639d1c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.680114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 271/561] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:21.680114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.680114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.680114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.697512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:21.698894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:21.701044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.701875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.702161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.702434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.716151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.717650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:21.738656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:21.740328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 291/562] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:21.750193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.790824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.810084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 272/563] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:21.810084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:21.889966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b55589-7378-4b8b-8927-cec54738204d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.933060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:21.934079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:21.934389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:21.934671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:21.939427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:21.951070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:21.959132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:21.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:21.973600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:21.975506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:21.975506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:21.975506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:21.975506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:21.975925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db4908-9019-4207-9dbc-3e7903737af6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:21.976855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 292/564] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:21.979863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.013698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.014500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.014777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.015057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.022213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.023687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:22.038577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:22.039581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:22.041147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:22.042158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 273/565] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:22.043536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.078370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.099081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bea8-764b-46d8-9354-f9ea03a4c0cb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.140879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.141904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.142190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.142470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.143206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:22.143514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.144918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528595d-3404-431b-87a7-4461e0355376 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.160344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 293/566] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:22.161823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d5da-688a-41fd-917c-8fc85adf8b3e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.162115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d5da-688a-41fd-917c-8fc85adf8b3e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.162651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d5da-688a-41fd-917c-8fc85adf8b3e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.163967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 294/567] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:22.164692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d5da-688a-41fd-917c-8fc85adf8b3e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.169282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5106e-cde1-45c6-ad49-975383668fdf None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.206983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.207702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.207983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.208262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.216832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.224647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:22.233214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:22.234380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:22.235213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 274/568] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:22.235955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.307497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d5da-688a-41fd-917c-8fc85adf8b3e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.346073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9773779-334e-4af4-ab65-cb370d5997de None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.348306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.349053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.349358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.349651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.350420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:22.350728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.352108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.364622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 295/569] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:22.365492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.368360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.395512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.396546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.397014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.397503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.398664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:22.399176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.402876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58116ffd-1a18-42bb-9978-ad6e2f1406ad None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.421891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 275/570] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:22.423273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044072-06ce-4469-8824-454faf758117 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.423582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044072-06ce-4469-8824-454faf758117 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.423868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044072-06ce-4469-8824-454faf758117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.442843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f8044072-06ce-4469-8824-454faf758117 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:22.444069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f8044072-06ce-4469-8824-454faf758117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:22.506174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.511556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb52c4-d464-4586-911b-fa5eb37014d7 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.526465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 276/571] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:22.527344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8044072-06ce-4469-8824-454faf758117 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.553525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.554325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.554612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.554901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.563316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.568673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:22.584150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:22.589829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 296/572] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:22.592104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.653015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8044072-06ce-4469-8824-454faf758117 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.666147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.691286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.691949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.692233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.692507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.693246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:22.693598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.695087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.703662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 277/573] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:22.704462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.708691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c4836-2fa2-47ef-b36a-5bd320561b55 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.751266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.752556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.752999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.753544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.763812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.765210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.769095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:22.780695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:22.788431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 297/574] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:22.789647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.833884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cdf8f-476c-405b-bc9a-ec0f1b09f422 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.871262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.871935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.872232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.872510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.873378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=0c95e2d016a84b5a804f4d18816a81ee) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:22.873699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:22.875175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:22.893629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 278/575] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/role_assignments?user.id=0c95e2d016a84b5a804f4d18816a81ee&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:22.893629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:22.911907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2be0d-3fc5-4f54-b569-e567fa242316 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:22.954383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:22.967106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:22.967837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:22.968117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:22.968387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:22.980490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:22.988714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:22.990109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 298/576] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:22.991558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.053432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb1fbb-32e5-4e87-b3be-add037b185ce None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.074802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.103053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.104037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.104037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.104325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.114376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:23.114376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.114376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.127807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59184a7d-030b-48c1-bad0-4d1dee584434 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.133657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 279/577] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:23.135279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.176209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.177012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.177292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.177594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.194665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.197263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:23.217205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:23.218212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:23.219113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:23.219113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:23.219113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.219113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:23.219113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 299/578] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:23.219414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.271269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.284138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34cdb4-c367-4fb9-a55f-a4fb323e6ecb None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.326599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.327191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.327481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.327782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.328610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:23.328926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.330411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.349811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec1bbb-107c-45ff-8e7b-df3b1b8f8ca6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.355164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 280/579] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:23.356166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.398927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.402594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.403660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.404139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.404613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.405972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:23.406484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.408603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.423187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 300/580] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:23.424769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.523115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5bdce-c781-4b90-9b08-28b3da0509bc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.562853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.563656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.563940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.564218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.568888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.576315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.581448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7766-c488-4ca4-8d7b-8a2632274978 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.581581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:23.596758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:23.597892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:23.598775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:23.598775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:23.598775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.598775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:23.598775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 281/581] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:23.599053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.622856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.623693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.624060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.624558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.625312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:23.625657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.627053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.641661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 301/582] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:23.642512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.695459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.719553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045b9cb-cdd1-4ba5-a228-ba4ddc3b09a0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.768705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.769556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.769842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.770121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.771536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ba78-4a99-456a-9f1a-b160ab2d2b0c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.781657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.783141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:23.802551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:23.803669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:23.804554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:23.804554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:23.804554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.804554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:23.804554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 282/583] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:23.804831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.816117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.817128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.817598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.817873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.830557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.833167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:23.851752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:23.852955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 302/584] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:23.855202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:23.867528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.940906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7a3b6-4f57-4c94-9486-0673f20b58f2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.978311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:23.978977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:23.979361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:23.979529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:23.980171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40d2d-5966-449e-a327-45d0b3885b51 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:23.980385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:23.980540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:23.981963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:23.984383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:23.994893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 283/585] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:23.995631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.023058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.023906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.024194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.024472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.032748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.034244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:24.049478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:24.050470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:24.051385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 303/586] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:24.051966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.120774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf4ded-4a5a-4f25-a588-9223a0ae3002 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.123395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.157274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.157976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.158262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.158534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.158970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ff207-44c1-49d0-a598-0ca591f8b0e4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.159446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:24.159535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.160935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.175358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 284/587] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:24.175358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.207884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.208565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.208850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.209127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.209893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:24.210203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.211591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.224464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 304/588] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:24.225320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.251735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.315334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be86e2a-bd38-4c72-8a1c-195bb21b3d1e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.353918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.355286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.355570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.355846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.362903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.364360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.371293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82083c8-6e96-4593-bf79-6f638bd11b54 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:24.378338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:24.379234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:24.380232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 285/589] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:24.380735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.407652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.417741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.419410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.419410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.419410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.420008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:24.420324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.421815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.449841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 305/590] 10.222.0.42 () {56 vars in 1139 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:24.449841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.506828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9e108-444b-49f7-8cee-f4c152fbb5d8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.521599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.543604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.544462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.544777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.545041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.552393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.554318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.566147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071da03b-94dd-44a0-af36-801622a25f67 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:24.570183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:24.571383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:24.572526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:24.576360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 286/591] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:24.577271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.612269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.613166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.613418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.613651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.621272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.622818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:24.637575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:24.638578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:24.639506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:24.643465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 306/592] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:24.644365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.693756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.695469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7718219-354f-4503-b0fd-d2e15f26d842 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.733154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.734116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.734563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.734839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.735630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:24.735938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.737429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.754157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 287/593] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:24.754157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.757061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52410b0-882b-49a7-a14f-70e0efdaa938 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.787731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.797508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.798317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.798601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.798876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.811038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.812528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:24.831385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:24.832505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:24.833553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:24.833553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:24.833553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:24.833553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:24.833553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 307/594] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:24.838882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.864866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acddf8-8211-4e14-98e7-8f45988b0dfd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:24.914533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:24.915468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:24.915920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:24.916392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:24.917770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:24.918279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:24.921251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:24.946051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 288/595] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:24.946051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:24.956190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:24.961388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9694ba-c064-461e-8102-15d6e9e05345 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.002098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.003824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.003824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.003980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.005735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:25.005735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.007123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.030504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 308/596] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:25.030700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.069061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.079195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bcca-0d55-4c7e-aa23-a581edd076c0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.114543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.115437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.115719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.115996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.132940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=cb545deab49645c5947c3eacd618dc07, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.135244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.147814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55511e66-c4b7-4694-81b2-52a1384208a6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.184846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.185508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.185742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.186017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.186734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:25.187040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.188414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.198542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 289/597] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:24 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:25.208087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 309/598] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:25.208087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.209392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.237164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.348093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540db8-ae92-4b90-8d80-9bafb98bfd98 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.348971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86c48-a10b-40e9-99a5-e1d608d72909 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.389025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.389866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.390154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.390432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.393084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.394130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.394311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.394581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/e7017c7303f94b6380b99cb0b09d7e56/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.407231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.408865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.412887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.414534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=e7017c7303f94b6380b99cb0b09d7e56, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.416531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462500e-24da-4d46-9943-f125e8b51ba8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:25.428861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:25.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 290/599] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:25.431143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.506272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 310/600] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:25 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/e7017c7303f94b6380b99cb0b09d7e56/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:25.512398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161f6c1-232a-45e4-ae72-7fa4fcd76db0 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.512701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161f6c1-232a-45e4-ae72-7fa4fcd76db0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.512983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161f6c1-232a-45e4-ae72-7fa4fcd76db0 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.514539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 311/601] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:25.515047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161f6c1-232a-45e4-ae72-7fa4fcd76db0 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.558212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f3d-d028-4ca7-8bb4-45907732683f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.607727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.608538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.608833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.609106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.614315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.619719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.621479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-cf053778-e81e-4b26-b1c5-527a9db63e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:25.643145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:25.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:25.644958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:25.650549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 291/602] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:25.652589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.652882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.653165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.672313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161f6c1-232a-45e4-ae72-7fa4fcd76db0 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.674425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:25.676236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:25.723487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.724545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.725026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.725517 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.726426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:25.726932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.729650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.742109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 312/603] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:25.743180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.748533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.756539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 292/604] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:25.762889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.866876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc00e6-7a86-4bf4-959d-7819147296f9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.876817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380c26d-d77a-43e2-9e1a-5b215821d167 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:25.901772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:25.913987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.914358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:25.916461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.916662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.916846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:25.917003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.917523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:25.918197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:25.919413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:25.919935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.922121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.924706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:25.926488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:25.939220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 313/605] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:25.940301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:25.948654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:25.949832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:25.950961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:25.957131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 293/606] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:25.957131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:26.009987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:26.077053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d7d26-e03b-4146-89a7-3b65314ecbd8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.091343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c94c0-b312-47e6-8a28-99d547846313 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.118708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.119887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.120926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.121398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.136389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:26.136684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.137765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.138150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.138575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/5e62e836ab6e4f42814b3151bfe686d2/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:26.146231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:26.147458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:26.148469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:26.151081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 314/607] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:26.152068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:26.164758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=5e62e836ab6e4f42814b3151bfe686d2, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:26.167039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:26.218921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:26.271204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 294/608] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:25 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/5e62e836ab6e4f42814b3151bfe686d2/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:26.272493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:26.295185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5843d-ebbf-47d3-9180-b39ef4fa5b53 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.343706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.344608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.345042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.345424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.346689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:26.347078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:26.349299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:26.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 315/609] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:26.373045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:26.491161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:26.503701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ad39c-de93-44bc-84fa-09fb3303231e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.513686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51dcbec-371d-44c2-922f-a11ba795160b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.534215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:26.553303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.554212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.554491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.554765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/2be41ec2de3948e6a015dfe35c47a0a8/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.573814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=2be41ec2de3948e6a015dfe35c47a0a8, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:26.576948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:26.580071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.580766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.581053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.581332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.582764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:26.582764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:26.585028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:26.597888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 295/610] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:26.601098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:26.668646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 316/611] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:26 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/2be41ec2de3948e6a015dfe35c47a0a8/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:26.766953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c00649-e55a-4456-9b92-6e174ccec470 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:26.838219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:26.839006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:26.839299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:26.839581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:26.855616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:26.857477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:26.885305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:26.886288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:26.887141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:26.887141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:26.887141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:26.887141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:26.889176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 296/612] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:26.898493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.041953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa5a8c-030b-4a1e-88f5-8dbf50addea8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:27.077908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:27.078574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.078859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.079132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.079925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:27.080233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:27.081651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:27.094850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 317/613] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:27.100139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.116514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:27.222502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659ab84-f42e-4730-aaa1-395861e6f465 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:27.266039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:27.266838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.267121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.267397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.275744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:27.277202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:27.296767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:27.297802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:27.298714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:27.302454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 297/614] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:27.307875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.341119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:27.467760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9db37c-ff8b-4761-85a6-84ea53f60f86 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:27.514949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:27.515638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.515924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.516197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.517042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:27.517366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:27.518820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91c902-84c6-4a1f-a69d-430fd7a8f655 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:27.546530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 318/615] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:27.551559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.598747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea9ad0-be95-4640-b386-183e8346526c None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.599052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea9ad0-be95-4640-b386-183e8346526c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.599307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea9ad0-be95-4640-b386-183e8346526c None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.601145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 319/616] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:27.606916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.607204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.607480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.635413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:27.636841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:27.660367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:27.716843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730c8d4-09bd-4acd-8cee-bc5c2ac41eca None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:27.767376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 320/617] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 16:07:27.794790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:27.796122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:27.796525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:27.796804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:27.819807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:27.821970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:27.827151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.830746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:27.842411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:27.843324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:27.844921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:27.844921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:27.844921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:27.844921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:27.844921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 298/618] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:27.854399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:27.957678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ed889-2a0c-4bc7-bfd9-846573c4b4e6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.002101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.003266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.003831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.004345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.016775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-fb8cd2dd-6581-4151-9ca0-7dbc05a63491 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:28.025713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:28.026646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:28.027502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:28.030542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 321/619] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:28.032192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc804-2de5-4af7-aa01-9c6a655b62a3 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.032671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc804-2de5-4af7-aa01-9c6a655b62a3 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.033150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc804-2de5-4af7-aa01-9c6a655b62a3 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.035263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 322/620] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:28.036144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbc804-2de5-4af7-aa01-9c6a655b62a3 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.049438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbb19e-9449-4e73-a679-52a0788692ef None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.075001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.090336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.091569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.092582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.094100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.095324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:28.096194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.097823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52068ab-264f-4675-ad12-b13c8fbe68f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.115815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 299/621] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:28.117259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.117916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.118602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.119724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.142083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:28.143612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:28.164976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbc804-2de5-4af7-aa01-9c6a655b62a3 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.206528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.207567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.208053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.208522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.209678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:28.210219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.212469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.218193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 300/622] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:28.219301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.223159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 323/623] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:28.234389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.332264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a256f-c52b-45e0-aa9a-e3ccb982682a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.345901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086669f-7ed1-46be-ae5c-f3eb6142ac31 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.346367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.367859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.369191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.369670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.369951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.375907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.378558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.380038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.386722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.387839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.388331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.388811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:28.395430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:28.404067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 301/624] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:28.405189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.406248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.406248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:28.424434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:28.425545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:28.426436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:28.426436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:28.426436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:28.426436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:28.427054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 324/625] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:28.428193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.517456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d76c03-81a6-4092-a903-7df8cee922e9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.532446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8deb05b-6df0-4d32-9e0a-b859ca0008e2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.555143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.557271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.557823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.558272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.568724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.569799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:28.570126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.570578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.571022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.572775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:28.573306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.574967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:28.579766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:28.580677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:28.581530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 302/626] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:28.582225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.595518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 325/627] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:28.597064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.637747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.704034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686c712-b917-4d6f-b43c-924f9a0c7c3c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.745928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.746913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.747408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.747856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.748756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:28.749309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.751111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.765750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 303/628] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:28.766882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.769095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.834327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a71f4c-e4d4-4a8c-8295-75c05fd425fa None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.869021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:28.872654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.873478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.873869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.874172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.875049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:28.875431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.877245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:28.879912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf3e2-c134-4c77-b09e-e64e1a1e9f63 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:28.887367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 326/629] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:28.891321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:28.919061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:28.920476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:28.921004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:28.921530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/0c95e2d016a84b5a804f4d18816a81ee/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:28.943625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=0c95e2d016a84b5a804f4d18816a81ee, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:28.947063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.003374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.006606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d85ea-f12b-483b-88a2-335e6d7316de None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.020973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 304/630] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:28 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/0c95e2d016a84b5a804f4d18816a81ee/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:29.022381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.047730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.048790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.049184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.049561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.071903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.073056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:29.101938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:29.103006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:29.103945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 327/631] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:29.109152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.157477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e0ea7-7b61-4039-8817-33e20f833bfc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.202976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.204420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.204942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.205651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.220366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.222226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.224562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:29.244470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:29.245580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:29.246538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:29.246538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:29.246538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:29.246538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:29.248605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 305/632] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:29.257189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.305030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bf7c-2a9f-4a6d-8898-7d39cd3a3191 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.351774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.352660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.352965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.353408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.354284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:29.354603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.356678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37ba97-36e7-4146-9945-560a4a4c294e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.382883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 328/633] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:29.384953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de685fad-7be0-44a2-8226-c29dd625ed64 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.390009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e6937-3c51-4a46-bd97-0f90d3bad569 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.390317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e6937-3c51-4a46-bd97-0f90d3bad569 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.390598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e6937-3c51-4a46-bd97-0f90d3bad569 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.393230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 329/634] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:29.394050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e6937-3c51-4a46-bd97-0f90d3bad569 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.429525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.437565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.442835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371ae3d-7f1b-481d-a75d-c90ec2844f63 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.461195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 306/635] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:29.463443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.463875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.464177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.484816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:29.486055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:29.531042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e6937-3c51-4a46-bd97-0f90d3bad569 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.547482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.568984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.569769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.570060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.570403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.575413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 307/636] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:29.576392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.672427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.673945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.695866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.706659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc73845-f6ad-4460-aacb-1609cb2537cb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.744540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.745680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.746185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.746470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.754468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.755924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-74715c36-c136-408b-a243-9c4a7535b270 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:29.772806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:29.774145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:29.775064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 308/637] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:29.775718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56177374-233d-496b-bccd-4b1d6136e6bd None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.775718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56177374-233d-496b-bccd-4b1d6136e6bd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.775837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56177374-233d-496b-bccd-4b1d6136e6bd None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.781132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 309/638] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:29.781132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56177374-233d-496b-bccd-4b1d6136e6bd None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.800849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 330/639] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:29.801902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:29.835744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.896778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56177374-233d-496b-bccd-4b1d6136e6bd None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.924016 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203526ba-8028-48f6-974a-db8222403ae4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:29.938676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.939924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.940218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.940501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.949038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:29.961523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:29.967050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:29.967768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.968052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.968334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.969833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:29.970226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:29.971685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d1993240-0f8e-40e2-8929-f8d3108f18ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:29.972566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:29.973511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:29.974358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 310/640] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:29.975786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060083-7426-494b-bbed-dc335b01e395 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:29.975786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060083-7426-494b-bbed-dc335b01e395 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:29.975786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060083-7426-494b-bbed-dc335b01e395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:29.997983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-52060083-7426-494b-bbed-dc335b01e395 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:29.999348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-52060083-7426-494b-bbed-dc335b01e395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:30.002403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 331/641] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:30.007994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.097363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 311/642] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:30.097363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52060083-7426-494b-bbed-dc335b01e395 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.128727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb357dbf-86dc-4329-a5dd-25d26db96a10 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.139368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.165907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.166695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.169015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.169015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.178206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.179671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:30.198179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:30.199448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 332/643] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:30.200684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.219703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52060083-7426-494b-bbed-dc335b01e395 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.232036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.261368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.262214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.262907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.262907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.263897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:30.264426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.266053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.275378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 312/644] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:30.276308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.317217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7faa7-db05-4ce8-9918-90d4629e773d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.369124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.369837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.370183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.370651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.386295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.392146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f36e4b-7e0e-4266-b467-adcb8bee62d1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.428052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.429018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.429442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.429810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.441554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:30.447207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.448804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:30.450019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:30.451098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 313/645] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:30.456985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.488674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.557572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd42424-7e2b-49d6-b133-eb1746c0ca33 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.569170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 333/646] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:30.574997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.592952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.593643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.593931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.594205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.594916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:30.595223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.596590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.613374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 314/647] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:30.613374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.636646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.691476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8222ce-d4c8-4f19-be16-7b7b228a0c97 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.718715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.724797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf8427-d998-42b5-9c59-0b94446d2334 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.726935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.728524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.728918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.729298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.740129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.741671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:30.759241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:30.760416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 334/648] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:30.762019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.769920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.770604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.770893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.771173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.772363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:30.772680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.775436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.789025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 315/649] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:30.789025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.874173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8368217-563a-49cb-9de4-8de85085298e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.876544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:30.909715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8968-0aa9-4e84-9447-7425356de6dd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:30.919043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.920417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.920810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.921192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.945109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.948205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.958552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:30.959419 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:30.959634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:30.959942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:30.967082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:30.968064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:30.968987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:30.970527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:30.970527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:30.970527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:30.970527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:30.970759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:30.970831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 335/650] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:30.979062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:30.989282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:30.990231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:30.991133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:30.996341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 316/651] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:30.997272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.018221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.105085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46428-d1a2-4e18-afd0-bc039ea0eb73 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.138556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f935dd1-43d0-443c-968c-6d2c27076e63 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.151788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.152670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.153056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.153481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.154342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.154795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.156333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.170285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 336/652] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.171321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.200420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.201246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.201590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.201876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.209017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.215722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.217393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:31.235813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:31.236887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:31.238070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:31.238070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:31.238070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:31.238070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:31.238862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 317/653] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:31.239717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.325882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.339054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a17588-9225-4c9b-8448-b0f0c2c342f8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.361906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529fd77-6ab2-4b9b-ac7c-e6e975d74048 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.377180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.377876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.378169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.378454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.379172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.379486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.380862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.392926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 337/654] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.393750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.396295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.396953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.397241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.397553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.398331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.398640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.400104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.411882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 318/655] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.412684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.464488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.496070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c64fe7-d1fb-4038-8fb4-12659914dd0a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.514790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02417-f95a-4539-b750-0b5aa3d26e64 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.529596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.530393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.530681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.530959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.538682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.540691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.549430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.550090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.550387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.550648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.551361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.551670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.553023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-779f9e47-8878-4993-aca3-2f9e7e0b5bd4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:31.555190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:31.556979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:31.557954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 338/656] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:31.558553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f774ad-7601-49bb-a35d-ac368b4b8ac4 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.558553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f774ad-7601-49bb-a35d-ac368b4b8ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.559128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f774ad-7601-49bb-a35d-ac368b4b8ac4 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.564099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 339/657] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:31.564099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f774ad-7601-49bb-a35d-ac368b4b8ac4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.572009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 319/658] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.572009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.576150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.662903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f774ad-7601-49bb-a35d-ac368b4b8ac4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.670674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e56570-0d64-4d0c-a206-a1e916fa591c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.682600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.697498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.698285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.698570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.698844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.705476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.705617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.706228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.706514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.706792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.706933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.707614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.707931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.709500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.715970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 320/659] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.716747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b318874c-5ffc-48f4-a2ee-da19db0462e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:31.720863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:31.722015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:31.723046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 340/660] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:31.726934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.727770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.728118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.746455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:31.747766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:31.826812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 341/661] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:31.828010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.830652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:31.858160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d80116-b485-412a-b209-b9fe13e750c8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.908655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:31.909332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:31.909639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:31.909912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:31.910743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:31.911051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:31.912741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:31.935318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 321/662] 10.222.0.42 () {56 vars in 1139 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:31.936712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:31.964664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296214f9-0101-4aaf-bcd2-e65a69a3177f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:31.981248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.006268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.007510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.007950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.008360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.016687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.018282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:32.033195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:32.034492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:07:32.035642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:32.036915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 342/663] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:32.037676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.065272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf508b9-04a1-42cc-8aa2-f18e5dea8853 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.102158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.102953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.103273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.103661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.104548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:32.104958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.105685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.106648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.119789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 322/664] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:32.120762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.155117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa723-95b3-417c-9329-31cc8ef197b6 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.193484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.194383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.195008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.195008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.206083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:32.215032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:32.216211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:32.217249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:32.221199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 343/665] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:32.222086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.223755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.233984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282189b-c0b1-4793-95ff-c6afaf9ae205 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.270559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.271142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.271428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.271706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.330952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebe948-f0e2-4db0-98c2-5721e0353b80 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.342213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.343674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.356326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.368729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.369519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.369862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.370208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.371120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:32.371503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.373000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.398176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 344/666] 10.222.0.42 () {56 vars in 1137 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:32.398176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.443835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.455225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 323/667] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:32.455225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.508603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197bd5f-1d2f-457d-9c3e-faedcd7c67fe None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.544821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.545739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.546187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.546472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.547308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=e7017c7303f94b6380b99cb0b09d7e56) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:32.547630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.549082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.554999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 345/668] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/role_assignments?user.id=e7017c7303f94b6380b99cb0b09d7e56&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:32.555935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.578998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f1de-734e-42cb-bd2d-a69120be557c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.642233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.643578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.644190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.644804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.646255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:32.646890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.649096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.650461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.672383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 324/669] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:32.672383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.739880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:32.745900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbc0b8-9aeb-4cec-96bd-5015da287904 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.785975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.786899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.787177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.787452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.806103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-2cbc-4f4d-9e2a-c1881ab557d5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:32.853293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:32.854640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:32.854934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:32.855211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:32.867786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.869271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.882278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:32.884358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:32.886771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:32.887835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:32.888870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:32.888870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:32.888870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:32.888870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:32.888870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 325/670] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:32.893846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:32.950029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.010421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 346/671] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:33.012211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.033385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b84f7-907b-4f28-a468-f2986e14f0fd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.043515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.080608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.081476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.082094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.082447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.097044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.099141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:33.123628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:33.124558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:33.125574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:33.125574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:33.125574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:33.125574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:33.125574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 326/672] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:33.126437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.127540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7cfea-0ad2-4df1-9305-dced50e90370 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.167212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.168176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.168633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.169108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.170069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:33.170736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.172407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.186253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 347/673] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:33.187384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.192070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.268626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac291df-53dd-49ea-958f-db28cbf96d4a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.309371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27986159-70a7-4276-bc49-8d12d914fceb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.315040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.315833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.316116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.316387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.331144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.333032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.347697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.348684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.349182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.349686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.350754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:33.351273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.353599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:33.355528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:33.356401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:33.357268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:33.357268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:33.357268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:33.357268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:33.357268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 327/674] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:33.357974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.369453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 348/675] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:33.370559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.375752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.438003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.481794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9c10d-e47f-4a86-aa68-c9112d61fdcd None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.491898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c526e12-01ab-4bf0-b5a9-91731653967f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.523045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.524484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.524777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.525160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.526564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:33.526877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.528767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.534709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.535793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.536345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.537132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.543535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 349/676] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:33.544459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.545976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.548090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:33.564417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:33.565361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:33.566245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:33.568812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 328/677] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:33.569843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.647324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.674436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abe6c7-1b3c-4147-a45d-85fd4eeb2a6c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.684559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3c2e9-7894-49c7-990a-51487c38bf01 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.703796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.723397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.724878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.725314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.725778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.728451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.729374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.729756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.730141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.734792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.736556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.737590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.739138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:33.753980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:33.755149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:33.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:33.757709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:33.759220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:33.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:33.761818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 350/678] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:33.762770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.766654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 329/679] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:33.768526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.881194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41d0f2-4726-48fe-bbee-d4e14484948c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.897260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcce42-3d16-4787-bdcf-a0be42b46e34 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:33.918997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.919735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.920116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.920491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.921405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:33.921814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.923345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.935404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:33.936707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 351/680] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:33.937771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.949169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:33.952374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:33.952374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:33.952374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:33.952374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:33.952374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:33.954221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:33.968910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 330/681] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:33.970140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:33.996434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.066882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6e0fe-f7b6-42f7-a406-37d11e27339b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.085206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcdb25-49d0-497e-a1c9-33c00bebccc5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.104433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.105106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.105477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.105726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.106668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:34.106986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.108436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.120330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.120979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.121293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.121600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.122416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=5e62e836ab6e4f42814b3151bfe686d2) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:34.122726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.124136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbad2d-e938-4d21-a4cb-6530425ad4c7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.127092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 352/682] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:34.127816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 331/683] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:33 2022] GET /identity/v3/role_assignments?user.id=5e62e836ab6e4f42814b3151bfe686d2&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1ec3-1cff-47cf-8050-190f5de664c4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1ec3-1cff-47cf-8050-190f5de664c4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be1ec3-1cff-47cf-8050-190f5de664c4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 332/684] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:34.141503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be1ec3-1cff-47cf-8050-190f5de664c4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.215796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.268274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be1ec3-1cff-47cf-8050-190f5de664c4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.281687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88d5bc-8e8d-46ea-88c8-24902678890a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.283748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.315262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.316226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.316692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.317639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.335464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.338289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.338958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.339590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.341400 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=2be41ec2de3948e6a015dfe35c47a0a8) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:34.342101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.344974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45566634-7e06-4d09-bea9-12c65aea820e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.358984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 353/685] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/role_assignments?user.id=2be41ec2de3948e6a015dfe35c47a0a8&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:34.362818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.363466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.364128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.398821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:34.401770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:34.411961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.413881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.489825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.512855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 354/686] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:34.541575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 333/687] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:34.546055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.547178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.659052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab69ce-1019-43c2-8027-daf57297b9a1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.660638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.668674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80e422-4e00-417c-b8b8-6b83ba801e9d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.697816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.698622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.698908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.699179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.707985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.708781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.709061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.709334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.710027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.711703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.721209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:34.730426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:34.731389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:34.732640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:34.733584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:34.735582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:34.742181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:34.742181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:34.742181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:34.742181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:34.743429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 355/688] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:34.743684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 334/689] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:34.743929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.744123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.748319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.847227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9660f9-06a7-41a6-937d-15e34fdcb4c6 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.850853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0762f1-c3ed-48f8-ac83-19f76ff58931 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:34.885074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.885848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.886147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.886404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.887235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:34.887549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.888928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.890567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:34.891243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:34.891631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:34.891917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:34.892659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:34.892969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:34.894364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:34.901395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 335/690] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:34.906880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.909683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 356/691] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:34.918350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:34.967206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:34.983000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.021454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278608f-9844-4de6-bd4a-a21040237130 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.058935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.059741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.060027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.060311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.067769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.069245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:35.084774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:35.085855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:35.087301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 336/692] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:35.094486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.129849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8fde5-3ef8-418f-b8a4-913efa170fdd None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.202080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.202933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.203231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.203427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334959b-c102-4ce5-ae76-e8d51ba27e5d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.203562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.226759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.228376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.243244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:35.259994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:35.270313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:35.271199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:35.271199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:35.271199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:35.271199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:35.271418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.271418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.271418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.271418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.271764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:35.271764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.271897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 357/693] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:35.271897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.272042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7addeb8-1d60-4e8a-920d-2b94545b0e4a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.288156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 337/694] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:35.290713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e78af-afa3-40ab-9c9e-a0af71ba89d8 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.291151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e78af-afa3-40ab-9c9e-a0af71ba89d8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.291445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e78af-afa3-40ab-9c9e-a0af71ba89d8 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.297430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 338/695] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:35.298567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e78af-afa3-40ab-9c9e-a0af71ba89d8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.351191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.385726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce119b-ecea-4e4d-abf2-84a28a1ab34a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.421659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.422453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.423265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.423619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.424693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:35.425104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.427128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f2669-abf3-42cd-8155-4804209049bc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.437670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e78af-afa3-40ab-9c9e-a0af71ba89d8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.443012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 358/696] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:35.444467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.444836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.445190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.467703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:35.469194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:35.483711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.484639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.485004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.485691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.487081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=0c95e2d016a84b5a804f4d18816a81ee) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:35.487660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.489584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.499649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 339/697] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/role_assignments?user.id=0c95e2d016a84b5a804f4d18816a81ee&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:35.500469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.575157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 359/698] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:35.602182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.630434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.685838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.733611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a2fd-61a9-4a91-9380-95284c6722a7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.738721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74b65-fef6-4780-9c26-2fddc2733d1c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.772122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.773002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.773289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.773599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.778955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:35.779757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:35.780045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:35.780326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:35.783408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.784869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.793457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:35.795161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:35.800801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:35.802898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:35.804575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 340/699] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:35.816695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:35.818802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:35.821091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:35.821091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:35.821091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:35.821091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:35.821091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 360/700] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:35.821802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.824859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:35.960676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94debf-7c6a-41d4-b350-61959467c15b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.961076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce31b39-ed99-4a9f-b98c-6c17e9d2e9f3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:35.975553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:35.977627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:36.004618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.005415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.005742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.006228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.006563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.007448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.007673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:36.008637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.008870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.010756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.011263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.013505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:36.013505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.014941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.028027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 361/701] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:36.034275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:36.034696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 341/702] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:36.040961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:36.152744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f930d5b-27fe-4c47-be5a-8a7059f12b99 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:36.158263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f1aea-474f-485c-b948-58030350e8de None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:36.189181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.189795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.190080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.190353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.241899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:36.262345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.263871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.277818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.278329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.278618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.278896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.279628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.281118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.368122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 342/703] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 16:07:36.373633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:36.394735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 362/704] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:07:36 2022] POST /identity/v3/users => generated 254 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:36.456535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:36.485806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a12c44-5cee-46c0-9d59-d8c57883a45b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:36.523150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.524647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.524972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.525247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.536931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.538446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:36.558737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:36.559707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:36.560641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:36.560641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:36.560641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:36.560641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:36.560641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 343/705] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:36.565704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:36.596943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:36.694508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeff06d-66b5-4b09-9592-82f8cd25de6f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:36.733631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:36.734273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:36.734560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:36.734837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:36.735563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:36.735870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:36.737250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:36.756541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 363/706] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:36.762430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:36.936927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d9d21-1e14-47da-811f-975fef247aa3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:36.994020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.003529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.004857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.005382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.005877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.018974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.021700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-a3d512fe-ca4e-4a0b-9bb4-864713edee30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:37.048922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:37.049986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:37.050908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:37.051795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 344/707] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:37.060384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.157768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b191b2-ca90-4205-9845-964cc45a7cc5 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.158060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b191b2-ca90-4205-9845-964cc45a7cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.158349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b191b2-ca90-4205-9845-964cc45a7cc5 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.159850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 345/708] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:07:37.167209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60650-d3f8-4dbb-9949-faf03b1b29b1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.167766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60650-d3f8-4dbb-9949-faf03b1b29b1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.168149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60650-d3f8-4dbb-9949-faf03b1b29b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.191489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5dc60650-d3f8-4dbb-9949-faf03b1b29b1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:37.193152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5dc60650-d3f8-4dbb-9949-faf03b1b29b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:37.207659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.221147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bfd73-857f-42f9-a830-7cc3cb03e352 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.258861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.259481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.259771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.260050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.260839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:37.261149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.262640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c95733-c71c-4ba5-a9b2-272891662fbc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.276422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 364/709] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:37.277835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18309b29-3781-4fa8-8dcb-6f7137a17f72 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.278127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18309b29-3781-4fa8-8dcb-6f7137a17f72 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.278405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18309b29-3781-4fa8-8dcb-6f7137a17f72 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.279760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 365/710] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:37.284356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18309b29-3781-4fa8-8dcb-6f7137a17f72 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.290510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 346/711] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:37.292182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.292676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.293152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.313303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:37.315184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:37.420137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 347/712] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:37.421410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.445034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18309b29-3781-4fa8-8dcb-6f7137a17f72 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.472854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.486646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.489190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.489190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.489190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.489190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:37.489655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.492012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.501295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 366/713] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:37.502469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.525016 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.541180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a053cdb-b6fa-46dd-9a1e-56aab6e45388 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.589119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.591028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.591520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.592231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] PATH_INFO: `/v3/services/network` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.593823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.595472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:37.612945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:37.614159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:37.614997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:37.619399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 348/714] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:37.620811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.641030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c803aa-7bef-403e-8b5d-b9e1c404749c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.681211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.682145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.682424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.682705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.683440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.684827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:37.701701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:37.703092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:37.703925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:37.708904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 367/715] 10.222.0.42 () {56 vars in 1129 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:37.710353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.763790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6ba9c-c90f-487a-92a5-8bae918feb2a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.768112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.815091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.816135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.816623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.817102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.818043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] RBAC: Adding query filter params (name=network) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:37.819042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.827006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.837274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 349/716] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:37.845726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:37.847831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:37.866953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bec33-e2cd-4a71-ae90-941ef2805a72 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:37.908385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:37.910258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:37.910550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:37.910825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:37.912024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:37.912332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:37.915225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:37.931514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 368/717] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:37.935647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:38.045270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ebfe-d522-4f36-99b4-f3d8e684f8c9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:38.084986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:38.085751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:38.086126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:38.086456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:38.087157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:38.087472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:38.089151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46094cd0-344c-4f2b-b871-02bf752a8d51 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:38.099690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 369/718] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:38.143246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72919-6d6d-4b40-9d4d-161723c5db2d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:38.208990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad690e-307e-407f-926f-547756806a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:38.209709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad690e-307e-407f-926f-547756806a9d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:38.210001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad690e-307e-407f-926f-547756806a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:38.210283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad690e-307e-407f-926f-547756806a9d None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:38.211007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad690e-307e-407f-926f-547756806a9d None admin] RBAC: Adding query filter params (type=network) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:38.212031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad690e-307e-407f-926f-547756806a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:38.214126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad690e-307e-407f-926f-547756806a9d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:38.227747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 350/719] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:38.590341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e197c-1c08-4a11-9be2-3e91686eb634 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:38.591055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e197c-1c08-4a11-9be2-3e91686eb634 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:38.591570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e197c-1c08-4a11-9be2-3e91686eb634 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:38.593429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 370/720] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:38.598610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:38.599133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:38.599618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:38.622496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:38.624073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:38.659327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:38.719579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 351/721] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:38.757202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e197c-1c08-4a11-9be2-3e91686eb634 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:38.762046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:38.892899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e197c-1c08-4a11-9be2-3e91686eb634 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:38.940849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:38.941688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:38.941971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:38.942247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:38.953315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:38.965578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:38.966744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:38.967556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:38.969835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 371/722] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:38.975718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:39.130033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fd682-7c65-4498-906b-b7365f121a09 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:39.183041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:39.183953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.184385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.184826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.185757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:39.186228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:39.187761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:39.201391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 352/723] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:39.206715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:39.297767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:39.355568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22dee4-1d50-4f08-af5f-9ca49a4eb8aa None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:39.408331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:39.409527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.410018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.410522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.434047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:39.435773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:39.456258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:39.457231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:39.458140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:39.458140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:39.458140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:39.458140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:39.460296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 372/724] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:39.465479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:39.497700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:39.611847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba5f8b-b68d-4bc1-bd4d-98ee2783ba94 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:39.657445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:39.658126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.658413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.658695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.659433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:39.659746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:39.661415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e670b3-f9bf-4660-90fc-d4136f76f53f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:39.676912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 353/725] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:39.682702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:39.804399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cca22-1d33-4c78-b093-187863dc97a4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.804591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cca22-1d33-4c78-b093-187863dc97a4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.804971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cca22-1d33-4c78-b093-187863dc97a4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.807081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 354/726] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:39.837370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.837767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.837927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.877793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31b1d2-fac1-44db-ad88-8a244ff872c4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:39.922397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:39.922963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:39.923636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:39.956845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:39.957703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:39.957996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:39.958282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:39.973120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:39.974635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:40.003645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:40.005799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:40.008087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:40.014556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 373/727] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:40.018642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.080855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 355/728] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 16:07:40.118563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.135081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.193200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73abeac-0017-4893-a4c5-681fdf89c826 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.233236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f84ad3-a97b-492c-b8a1-0b5b77241936 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.274806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.276028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.276517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.276994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.283265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796d62-feca-4d6b-9771-0266060046be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.283836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796d62-feca-4d6b-9771-0266060046be None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.284177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796d62-feca-4d6b-9771-0266060046be None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.284397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796d62-feca-4d6b-9771-0266060046be None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.285302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796d62-feca-4d6b-9771-0266060046be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:40.285664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796d62-feca-4d6b-9771-0266060046be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:40.287376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796d62-feca-4d6b-9771-0266060046be None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.291850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:40.300193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:40.301214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:40.302124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:40.305513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 356/729] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:40.318360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.327091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 374/730] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:40.328423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796d62-feca-4d6b-9771-0266060046be None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.412782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.413803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.457394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db5c60-3c33-4f5a-bac5-d8ba4a558699 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.493698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796d62-feca-4d6b-9771-0266060046be None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.530486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.531169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.531462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.531741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.532506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:40.532817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:40.535255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.538547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.540041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.540759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.541045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.560792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 357/731] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:40.571686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.647264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:40.649461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc62b1-bd2b-4153-b31a-82e64e2c8251 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.692140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.700543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a55f5d-9139-45b5-b67b-1811f3f13dbf None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.739963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.741004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.741296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.741607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.746234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.757610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:40.760339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.782987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 375/732] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:40.784707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:40.785811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:40.787204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:40.787204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:40.787204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:40.787204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:40.788365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aed2a-04d7-4a3f-a6ca-a5bbd5ed2e2a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.788882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aed2a-04d7-4a3f-a6ca-a5bbd5ed2e2a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.789415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4aed2a-04d7-4a3f-a6ca-a5bbd5ed2e2a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.789821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 358/733] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:40.790818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.796024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 376/734] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:40.797496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4aed2a-04d7-4a3f-a6ca-a5bbd5ed2e2a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:40.923280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4aed2a-04d7-4a3f-a6ca-a5bbd5ed2e2a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.939478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376d826-f7eb-4c3c-8032-ea65ffa56eb9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:40.964738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.965354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.965664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.965946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.966679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:40.966990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:40.968389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b75d0-ab1c-4f21-9754-6d9edc4ed6ed None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:40.970222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:40.981645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 377/735] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:40.985925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a463-2344-4938-bf0e-23ed655ba5a3 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.985925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a463-2344-4938-bf0e-23ed655ba5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.985925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a463-2344-4938-bf0e-23ed655ba5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:40.992952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:40.993805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:40.994084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:40.994363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.002423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3744a463-2344-4938-bf0e-23ed655ba5a3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:41.003637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3744a463-2344-4938-bf0e-23ed655ba5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:41.007002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.008571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-cc4efea7-d577-42d8-9b6a-ed545f919ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:41.028052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:41.029064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:41.030045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:41.030045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:41.030045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:41.030045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:41.030045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 359/736] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:41.030815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de744a0-c1b6-4e22-887f-f348b8da147d None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.031105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de744a0-c1b6-4e22-887f-f348b8da147d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.031392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de744a0-c1b6-4e22-887f-f348b8da147d None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.032903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 360/737] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:41.033643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de744a0-c1b6-4e22-887f-f348b8da147d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.078264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.085718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 378/738] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:41.088110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cc17-0beb-4ac7-99ad-909ee7830a1e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.088522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cc17-0beb-4ac7-99ad-909ee7830a1e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.088813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6cc17-0beb-4ac7-99ad-909ee7830a1e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.094240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 379/739] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:41.094240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6cc17-0beb-4ac7-99ad-909ee7830a1e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.145205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de744a0-c1b6-4e22-887f-f348b8da147d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.182922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.183957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.184430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.184899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.198091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.200486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.212305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6cc17-0beb-4ac7-99ad-909ee7830a1e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-af200786-d374-41c2-afb7-9ad10ec2deb0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:41.225481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:41.227022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:41.228121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 361/740] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:41.229635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.229983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.230331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.247763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:41.248962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:41.249476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.263137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.264096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.264385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.264676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.265424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:41.265745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.267127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69d4b0-7a8c-4054-8aa4-49bcce1c2997 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.279800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 380/741] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:41.281084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.281408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.281692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.299178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:41.300385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:41.325692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 362/742] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:41.326559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.360445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.384069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 381/743] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 16:07:41.384920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.441727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772911e-6072-4c85-bf32-ab43d9045988 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.551054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e97228-4cf7-4bd7-8bee-b367190c5a84 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.577892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.598779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.602245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.602245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.602245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.602245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:41.602245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.603161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.611202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.612499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.613002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.613528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.620297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 382/744] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:41.621117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.631327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:41.640550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:41.641615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:41.642452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 363/745] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:41.643163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.729540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22541-d2fe-4a78-9c3a-64ea3c03c7f0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.740065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.749989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3e7d-d3f2-42fb-bd22-8cc534f2ddc2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.763487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.764455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.764710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.764867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.771677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.773127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.784627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.785294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.785611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.785894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.786875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:41.787570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:41.788398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:41.789741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:41.790519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:41.790603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 383/746] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:41.790603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.825096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:41.883409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 364/747] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:07:41 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:41.884244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:41.903267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403a10d-0e75-4a46-a84d-cb1fe823f91c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:41.957290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:41.958019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:41.958279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:41.958554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:41.959290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:41.961258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.026262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dba1a-84d0-4e7f-aae7-7687b29d656b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.027496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 384/748] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:07:41 2022] POST /identity/v3/services => generated 240 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:42.030827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.089564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.090147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.090444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.090717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.119847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.187634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.216793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d150-7d60-435f-962d-089ad9636619 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.217659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.219380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.273069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.274790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.275025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.275521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.276920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:42.277534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.280098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.295049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 385/749] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:42.296575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.357367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 365/750] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:07:42.358253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.421135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.437994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657fec3-22f2-4eb3-a27d-cf071a892256 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.474518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.475188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.475478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.475767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.476755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:42.477073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.478585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.495846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d43e30-b7c9-4721-aa39-bb457fad2847 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.497817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 386/751] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:42.499333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.513737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.539588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.540379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.540826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.540919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.552886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.555007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:42.573119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:42.574136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:42.575030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:42.575030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:42.575030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:42.575030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:42.579075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 366/752] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:42.579946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.619775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237944ca-946b-42d8-9075-7f3b6530d31b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.665962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.667032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.667810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.668105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.683142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.684642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.686795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.691863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8deee6-a786-4e4f-8445-d5466e25e6a5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:42.715001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:42.716153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:42.717242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:42.717242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:42.717242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:42.717242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:42.719710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 387/753] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:42.720709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.737991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:42.739140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:42.739285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:42.739695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:42.740639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=0c95e2d016a84b5a804f4d18816a81ee) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:42.741127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:42.743095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:42.758982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 367/754] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/role_assignments?user.id=0c95e2d016a84b5a804f4d18816a81ee&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:42.759841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:42.807190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:42.895951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17e46e-2963-490b-a099-bd0b177d48b3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.037463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.038123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.038420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.038703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.040070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:43.040392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.043217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9e8e-2f65-435a-98e1-4b63e79b910c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.046076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.075771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 388/755] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:43.092699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.099722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.100397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.100689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.100976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.102065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:43.102403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.103952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.141704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 368/756] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:43.142636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.193328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:43.313760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c7e1-281c-41e4-b900-73707a5c7fd9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.342642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:43.359170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.361031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b829d-b4c8-45e8-a65d-06db2921bdc9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.363431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.363719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.364002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.381288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.382892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:43.411190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:43.412223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:43.413159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:43.425833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 389/757] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:43.430436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.439676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.440487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.440823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.441107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.456675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.458281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:43.488485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:43.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:43.490398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:43.492806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 369/758] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:43.499209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.615368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5fbb1-81fa-4b4b-bb07-28796063fbad None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.657618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.658670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.659157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.659635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.660748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:43.661583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.663237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.684584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:43.688251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 390/759] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:43.693279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.735425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61ef08-2afd-48e7-8745-7760dd406389 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.750714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:43.816035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.816722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.817008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.817290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.818158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:43.818472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.820404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:43.848453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 370/760] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:43.855752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f47ca-d01d-4568-aa4a-ff40327d6cda None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:43.858104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:43.934685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:43.935510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:43.935803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:43.936085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:43.966221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:43.967831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:44.013755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:44.014681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:44.016170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:44.016170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:44.016170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:44.016170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:44.016170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 391/761] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:44.020361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.034135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3d7e1-7fdd-4791-97f9-b107af7398f7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.083778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.084571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.084947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.085300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.086839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ebdc88eed0ec4c19af2d34413c4f73e1) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:44.087359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.088989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.096057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 371/762] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:43 2022] GET /identity/v3/role_assignments?user.id=ebdc88eed0ec4c19af2d34413c4f73e1&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:44.173519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:44.184062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9377c-0059-4ea9-ad6a-9cd65b82793a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.229609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.230356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.230650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.230932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.231682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:44.231996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.233483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa4276-b3ae-498b-8ebd-08c90050b283 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.247625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 392/763] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:44.253015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.267248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:44.415152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a035b67-d71c-46d8-a169-555a0fe6af65 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.415497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a035b67-d71c-46d8-a169-555a0fe6af65 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.415719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a035b67-d71c-46d8-a169-555a0fe6af65 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.416511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28120-8afb-4c30-93de-0b43ce9b9b27 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.417364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 393/764] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:44.423303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.423596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.423882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.458442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:44.460199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:44.469370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.469370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.469370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.469370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.472620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.474246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:44.490102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:44.491048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:44.492422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 372/765] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:44.497326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.566595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:44.631312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 394/766] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:44.637291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ed888-8caa-4eec-8837-eaa88592626d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.666099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.678650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.680001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.680287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.680578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.681508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:44.681824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.683295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e002d-c15b-45ed-a21e-164807ffbce8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.697929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 373/767] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:44.699500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2379d0-3408-46ee-b011-3ac3447855d5 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.699793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2379d0-3408-46ee-b011-3ac3447855d5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.700075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2379d0-3408-46ee-b011-3ac3447855d5 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.706153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 374/768] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:44.706901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2379d0-3408-46ee-b011-3ac3447855d5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.763069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:44.781838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea6d61-ce78-4675-ba87-3293ab5442e4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.821209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.822114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.822522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.822891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.823006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2379d0-3408-46ee-b011-3ac3447855d5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:44.823722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.825418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-047cadd1-68bd-47df-b59e-1483e782e6c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:44.839098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:44.844523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:44.846586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 395/769] 10.222.0.42 () {56 vars in 1129 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:44.848658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.848658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.848658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.851159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:44.860529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:44.877311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:44.877311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:44.877311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:44.877603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:44.877603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:44.877764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=cb545deab49645c5947c3eacd618dc07, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:44.877764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:44.941431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 396/770] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:44.941431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:44.982477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 375/771] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:44 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:44.984455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.024793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.051539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dde3a3-017f-4aec-8b74-ef8fa6bd3a5f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.122249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b7f12-b7ad-4f3c-8c3f-7104c8b17e29 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.150957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.151561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.151849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.152127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.152861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.153161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.154600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.165186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.166615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.167106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.167598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] PATH_INFO: `/v3/services/network` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.168470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.170269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.175565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 397/772] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.181966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:45.186072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:45.187072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:45.187901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:45.191082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 376/773] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:44 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:45.198462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.277748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.325162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb9bf7-b733-4035-8ec3-3a1c4c8da194 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.345464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bd321-465f-4d1c-a15e-855b5476b110 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.370372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.371771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.372383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.373159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.374081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] RBAC: Adding query filter params (name=network) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.374695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.376863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2c52c-c909-455e-8c59-65e7a97cf46e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.384664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 377/774] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.386353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb7ef-f7f9-49de-aaf9-53201e4910b6 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.386826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb7ef-f7f9-49de-aaf9-53201e4910b6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.387282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb7ef-f7f9-49de-aaf9-53201e4910b6 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.388577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.393048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c373ba2-8105-4e15-aff6-0792915863e5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.393956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 378/775] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:45.395297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bb7ef-f7f9-49de-aaf9-53201e4910b6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.399229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 398/776] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:45.400730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.401096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.402321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.409676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.422570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:45.424183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:45.501715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 399/777] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:45.502987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.526710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bb7ef-f7f9-49de-aaf9-53201e4910b6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.544244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.568022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.568808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.569163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.569555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.570659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.570774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.572178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.577378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 379/778] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.578288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.662047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a388a8-7358-49d5-aafa-1e5417c2b27b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.691210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481324d-7b8f-4dfd-a558-0c809c360fe8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.708724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.709984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.711479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.711943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.712915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] RBAC: Adding query filter params (interface=public, service_id=99591824dc6743ada85dd95f08247970, region_id=RegionOne) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.713429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.715649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.718281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.727652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 400/779] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/endpoints?service_id=99591824dc6743ada85dd95f08247970&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.728811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.735851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.736704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.736991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.737316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.749068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:45.758641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:45.759583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:45.760436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:45.764798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 380/780] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:45.765787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.812781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:45.867191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a58d-adf3-4ea5-bf92-53d6ce326857 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.875605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c7cd3-c45c-4b3c-81bd-13261db437c7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:45.914282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.914793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:45.915193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.915478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.915755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.916419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:45.916524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.917014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:45.917522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:45.917850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.918734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] RBAC: Adding query filter params (interface=public, service_id=076e9dbd315e469985b6974f6f4d2d2e, region_id=RegionOne) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:45.919413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:45.921642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:45.928002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 401/781] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/endpoints?service_id=076e9dbd315e469985b6974f6f4d2d2e&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.929393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:45.929816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 381/782] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:45.938137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.009195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:46.059238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342efdb-82f3-426c-9c21-562e128c7148 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:46.102626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:46.103674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.104163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.104638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.105631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:46.106169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:46.107778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:46.119386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 402/783] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:46.127389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.213178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcba9f-8e63-494e-9997-c373e68bb7bc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:46.216145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:46.274788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:46.275462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.275744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.276019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.276701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:46.278124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:46.284840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 382/784] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:07:45 2022] GET /identity/v3/services => generated 772 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:46.290885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199693-1f62-4ce8-a9d3-4fc9b778ee99 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:46.319547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:46.333639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:46.334845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.335334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.335858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.348275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:46.351737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:46.372208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:46.373172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:46.374453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:46.374453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:46.374453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:46.374453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:46.376643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 403/785] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:46.385462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.537416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250bd9b-994f-45cd-a3ec-f5985a331f28 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:46.576514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:46.577178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.577486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.577766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.578485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:46.578791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:46.580166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53a79c-bf64-45b5-a908-77466081d567 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:46.593012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 383/786] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:46.598489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.709313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c770dcd-88c5-4837-90a2-1fcdd2d28eda None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.709913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c770dcd-88c5-4837-90a2-1fcdd2d28eda None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.710403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c770dcd-88c5-4837-90a2-1fcdd2d28eda None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.712450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 384/787] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:46.718343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.718932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.719392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.733368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656d6e-07b4-4a61-8081-2b90d61b18fc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:46.749796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:46.751396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:46.788181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:46.789700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:46.790100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:46.790392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:46.794039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:46.816146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:46.818435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:46.854266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:46.855278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:46.856206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:46.861208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 404/788] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:46.862015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.903925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 385/789] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:46.939986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:46.953844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.053838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d249886-561c-4dd4-a219-9d89375e91f1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.129402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.130089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.130375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.130673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.135163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:47.135466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.136986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.146913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deec25-7662-452a-bd93-f24a396ec7a8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.175036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 405/790] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:47.181440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.204463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.213376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.213376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.213376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.231011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:47.243264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:47.244259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:47.245099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 386/791] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:47.250995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.296163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.341946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0a50-5610-4804-8ade-30d16ac3ba6e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.345654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.389426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.390442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.390735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.391013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.405293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.407191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:47.428669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:47.430446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:47.432087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:47.432087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:47.432087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:47.432087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:47.432087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 406/792] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:47.442432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.563784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a21b57-8bad-42b4-86a0-8480f4c64879 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.576639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca7d97-0647-4781-8ba3-47b605286411 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.621537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.622337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.622635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.622961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.623663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.623799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:47.624077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.624844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.625322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.625658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd93ca-d72f-4316-b858-1737b0f423c9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.626041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.627268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:47.628039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.630156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e537dd-b48a-48a4-9a9a-9ba65cfafa49 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.637118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 387/793] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:47.639284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce77fb-6941-4f98-b21e-1fac08b479dc None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.639597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce77fb-6941-4f98-b21e-1fac08b479dc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.639943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce77fb-6941-4f98-b21e-1fac08b479dc None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.645404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 388/794] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:47.645404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce77fb-6941-4f98-b21e-1fac08b479dc None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.650852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 407/795] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:47.652607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.653112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.653623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.674087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:47.675854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:47.711346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.749438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.761815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 408/796] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:47.763039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.775917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce77fb-6941-4f98-b21e-1fac08b479dc None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.835013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.836243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.836684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.836977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.853207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.853870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:47.876077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:47.877170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:47.878283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:47.878283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:47.878283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:47.878283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:47.880587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 389/797] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:47.881931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:47.912565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521a085-cedb-48c4-b065-742a564038e6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:47.964151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:47.964960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:47.965250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:47.965813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:47.969657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:47.975407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:47.977533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:47.996466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:47.997652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 409/798] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:47.998975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.012656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f413e08-30cf-4f1e-a711-d78f65901126 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.050055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.058096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.058856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.059150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.063684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.073574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:48.082969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:48.084044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 390/799] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:48.085670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.128775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5dc1e-d692-496d-9ac3-8c760176dffc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.171064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.171753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.172038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.172318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.173363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:48.173670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.175233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.192541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 410/800] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:48.193333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.213264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc7289-5a81-48f4-9c6a-3f1020b00b3b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.256036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.257178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.258326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.259077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.260206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:48.260729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.262770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.263786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.278596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 391/801] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:48.279858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.323224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6792c44-b89a-41fa-a8a5-b052771d1e77 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.346446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.366040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.368818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.369639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.370135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.372277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:48.372872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.374555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.388754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 411/802] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:48.390399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.408156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c740b-209c-4ce1-841f-cc99f164559a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.453026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.453881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.454167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.454439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.465018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.467146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:48.488900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:48.490160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 392/803] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:48.492853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.536503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1604c-6346-4cd6-9786-30519fed7fb0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.566074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.591510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.593960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.594254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.594541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/0c95e2d016a84b5a804f4d18816a81ee/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.598687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=0c95e2d016a84b5a804f4d18816a81ee, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.601394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.630004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939eb34-30cd-4d74-a3cb-5959a698d8bc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.642802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.668228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.669101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.669411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.669697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.683484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 412/804] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:48 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/0c95e2d016a84b5a804f4d18816a81ee/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:48.685386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.687490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.691383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:48.728372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:48.729450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:48.730578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:48.730578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:48.730578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:48.730578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:48.730578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 393/805] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:48.735859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.853461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33658-da85-4b59-84f9-022efd94a38b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.893539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862d11d-7a92-48c6-93e8-060590d9ecee None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:48.895897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.896688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.897168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.897581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.900253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:48.900253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.901427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc1dc-adb5-421a-8e88-d0de9a6756a4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.924059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 413/806] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:48.925334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a055-dc2f-49f3-99e5-42fa15eac233 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.925676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a055-dc2f-49f3-99e5-42fa15eac233 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.925963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a055-dc2f-49f3-99e5-42fa15eac233 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.927395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 414/807] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:48.934747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.934965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.935188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.945020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:48.945668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:48.945950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:48.946324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:48.947064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:48.947373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:48.948933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:48.954521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:48.955742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:48.964032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 394/808] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:48.965900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:48.994989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:48.997334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.037706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 415/809] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:49.039043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.089510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a55230-fb58-4324-8d60-5cf741d260ac None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.128418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.129277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.129601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.129884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.143081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.144754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.151631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5474e-57fa-49ae-8555-afa3a4b9f91f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-460322a5-9d39-4076-8214-103c71035bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:49.165481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:49.166430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:49.167322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:49.167322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:49.167322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:49.167322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.167322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 395/810] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:49.168158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51da6c-2dc3-4197-bdf1-2f33b16143b5 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.168407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51da6c-2dc3-4197-bdf1-2f33b16143b5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.168692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51da6c-2dc3-4197-bdf1-2f33b16143b5 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.173486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 396/811] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:49.173486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51da6c-2dc3-4197-bdf1-2f33b16143b5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.198934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.199740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.200046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.200306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.208790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.210768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:49.227728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:49.228973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:49.229969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 416/812] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:49.230554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.259351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.275824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.309363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51da6c-2dc3-4197-bdf1-2f33b16143b5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.349595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.350647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.350990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.351925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.352600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.353063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50676da-0d81-486a-ae44-925344efa35e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.354020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-666e3c4b-8299-4de8-b432-c686d8a45c5f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:49.368592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:49.369811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 16:07:49.370852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.374697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 397/813] 10.222.0.42 () {56 vars in 1129 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:49.376017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.376320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.376601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.394257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:49.395894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:49.396003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.396688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.396979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.397258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.398481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:49.398910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.400310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.412703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 417/814] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:49.414211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.507925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 398/815] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:49.509046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.535870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.552369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b023d84-dd11-4bdf-94d8-66a0de94de2f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.591568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.592249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.592532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.592831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.593777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:49.594245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.594765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.596209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.619405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 418/816] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:49.620143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.629122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7170a66-1fa1-4d51-9172-9328fbb61794 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.667765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.668385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.668617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.668898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.669630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:49.669940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.671318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.675846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 399/817] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:49.676605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.735621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f37a9-b7c5-40ae-aa78-50f590f591bf None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.773101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.774224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.774688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.775185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.782178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.783745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.786281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4efb7-24e5-4f69-8e7d-1bd8aab92c31 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:49.798453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:49.799565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 419/818] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:49.800453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.805897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.825976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:49.830948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.831811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.832093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.832366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] PATH_INFO: `/v3/services/network` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.833013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:49.834478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:49.847181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:49.854020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:49.854851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 400/819] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:49.855622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:49.920713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231db9-cf3d-4c62-a705-af649d4eecec None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:49.958487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:49.959226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:49.959514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:49.959783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:49.969555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a76e65-d6dc-41ce-b588-af6cab747002 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.014136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.014847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.015139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.015424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.016141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:50.016454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.017886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.023258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 401/820] 10.222.0.42 () {56 vars in 1124 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:50.024286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.045491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.047025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.049859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.098919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.151418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d92566-f10b-4762-afec-1c5bc39500b0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.177488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 420/821] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:07:50.177488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.218414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.219462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.219938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.220390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.221768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:50.222278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.224751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.253415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 402/822] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:50.253415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.310320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66649a74-972c-44ad-b6fd-abcd033ccfd2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.313912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.353224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.353938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.354260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.354516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.355582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] RBAC: Adding query filter params (name=network) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:50.356224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.357913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.363418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 421/823] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:50.363418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.383506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aebec-016f-4270-8d41-1174123fe30e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.407360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.426896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.427617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.427846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.428122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.429191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.430927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.485524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0c58b-6678-47fc-a9ae-42b54af0a195 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.502045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 403/824] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:07:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:50.503014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.559047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.560090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.560453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.560812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.578538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.580549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:50.610962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:50.611973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:50.612873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:50.612873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:50.612873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:50.612873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:50.617879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 422/825] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:50.618886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.645936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.663660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8a89f-3e80-4720-b1ac-eec0984be877 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.707407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.708223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.708510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.708787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ebdc88eed0ec4c19af2d34413c4f73e1/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.725520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=ebdc88eed0ec4c19af2d34413c4f73e1, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.727539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.743836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215d9fd-fb86-4936-badd-6db038dedb01 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:50.784934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:50.785933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:50.786238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:50.786502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:50.787212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] RBAC: Adding query filter params (type=network) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:50.787524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:50.789589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:50.794822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 423/826] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:07:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:50.799573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.813146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 404/827] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:07:50 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ebdc88eed0ec4c19af2d34413c4f73e1/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:07:50.814734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:50.891397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.917594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:50.976736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4c1f7-6c14-4b3a-bc34-61d1794cd251 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.014672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aacae-c7a4-4df7-9096-b4210cd155a3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.058827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.060458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.060994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.061531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.062545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.064285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ee2cc-2b58-4253-933f-b7d25baf5b93 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.084438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.090977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b9216-81dd-4341-aade-95eaad2de301 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.106200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 424/828] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:51.107567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb07be-bdc1-4680-bfc4-3c3e91499a0d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.107858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb07be-bdc1-4680-bfc4-3c3e91499a0d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.108140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb07be-bdc1-4680-bfc4-3c3e91499a0d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.109464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 425/829] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:51.114326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb07be-bdc1-4680-bfc4-3c3e91499a0d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.149846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 405/830] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:07:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:51.158424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.159645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.159962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.197461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:51.201996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:51.321422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb07be-bdc1-4680-bfc4-3c3e91499a0d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.347658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 406/831] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:51.375821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.397633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.398721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:51.399078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.399457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.399739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.402958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:51.410664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.412743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:51.431499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:51.432537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:51.433534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 426/832] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:51.439093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.515399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd523587-76e1-4429-993a-2296ee6ca82e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.572105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508571b-4bff-4320-96fa-74dae1f8523e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.575068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.575883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.576169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.576449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.601692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:51.613188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:51.614189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:51.615023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 407/833] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:51.619463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.620144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.620429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.620716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.621550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:51.621865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.623653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.623778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.638145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 427/834] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:51.644742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.715093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:51.715753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:51.757513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213fdb2-ef2a-4546-939e-be9b7dfac7fc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.782295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5e40-7199-4fe7-9bda-08712c18cf3a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:51.798330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.799135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.799524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.799915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.800770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:51.801182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.803034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.817115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 408/835] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:51.819066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:51.828491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:51.829429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:51.829879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:51.830298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:51.831864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:51.832341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:51.834334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:51.849395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 428/836] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:51.950208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1d47d-1988-49c1-b63d-fa6e475d7e06 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:52.006005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:52.007922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.008672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.009241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.013002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:52.030542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:52.032941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:52.058967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:52.060020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:52.060911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:52.060911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:52.060911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:52.060911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:52.063337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 409/837] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:52.073614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:52.192731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab705ac-494d-49e2-a56f-143fb1b999c3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:52.223802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:52.233041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:52.233729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.234016 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.234298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.235165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:52.235476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:52.236951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:52.251147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 429/838] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:52.256331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:52.375852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8034b-9074-4767-8c5e-7b58661eba77 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:52.416903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:52.418215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.418724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.419247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.427804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:52.431455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:52.447852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:52.448852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 16:07:52.449856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:52.452530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 410/839] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:07:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:52.458152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:52.580648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23330188-8cf9-4151-8048-e344607ca20d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:52.619975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:52.620661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.620956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.621219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.622097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:52.622405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:52.623861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3ea02-0135-4dc6-a346-b6a4841eb53f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:52.647730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 430/840] 10.222.0.42 () {56 vars in 1135 bytes} [Fri May 27 16:07:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:52.653285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:52.707502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:52.770246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e1b4b-10dd-4915-9c42-1c52d7007f8e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:52.807669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:52.808667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.809137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.809638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.886930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:52.888727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:52.908647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785b39-93b8-4a40-94af-71aeb98c991c None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.908971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785b39-93b8-4a40-94af-71aeb98c991c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.909233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2785b39-93b8-4a40-94af-71aeb98c991c None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.911226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 431/841] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:07:52.918138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:52.918727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:52.919211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:52.941670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:52.948099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:52.949676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:52.992458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.008404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 411/842] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:07:53.014816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.045632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 432/843] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:53.067634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.145372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2744480-390e-43ea-b52d-57ceae42a336 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.184151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3afafd-0147-4d07-ac36-e05cf0a86518 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.190824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.192060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.192553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.193143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.204922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.206862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.215754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.224432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.232640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.232640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.232640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.232640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.232640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:53.233046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:53.233881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:53.234773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:53.234773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:53.234773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:53.234773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:53.235776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 412/844] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:53.241395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:53.246896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:53.248125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:53.249883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:53.254394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 433/845] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:53.255391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.317681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.370185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9cd5-1e7e-43a9-b657-efc58606b9c5 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.374292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd1924-a9cc-413a-9465-264e4cd22fd8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.410431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.411127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.411418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.411707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.412420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:53.412736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.413380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.414459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.414901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.415176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c31e8-d6eb-4f6e-9eee-e7f21d183c79 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.415498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.416478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:53.417050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.418570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd024a4-43c7-411c-81b5-836b5977449e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.419787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 434/846] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:53.421006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278123d-0e88-405d-9bdd-212aa61d6ff9 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.421294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278123d-0e88-405d-9bdd-212aa61d6ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.421604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278123d-0e88-405d-9bdd-212aa61d6ff9 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.425473 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 435/847] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:53.427080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278123d-0e88-405d-9bdd-212aa61d6ff9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.435873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 413/848] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:53.440518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.440518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.440518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.456722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:53.457947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:53.528167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.551038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 414/849] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:53.553545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.560114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.572706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278123d-0e88-405d-9bdd-212aa61d6ff9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.629786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.630609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.630906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.631194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.631988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:53.632325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.633806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.649375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 436/850] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:53.649375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.720490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57016907-08a2-4906-8d8c-0c759430e379 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.771768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.774357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.774650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.774931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.788488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:53.791538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:53.793200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8099ff-07af-4a6d-99d5-9f2106e98d9c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:53.812207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:53.813183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:53.814160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:53.822432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 415/851] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:53.825779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.847867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:53.849132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:53.849755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:53.850232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:53.863973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:53.872825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:53.874052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:07:53.875013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:53.881480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 437/852] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:53.883556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:53.890090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.918869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:53.969941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fb413-9ae0-472a-9c2c-28a596522b9a None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.018021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.018961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.019418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.019832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.021026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.021538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.023957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.027048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e297a6-cf8a-453f-8da2-1e66bc4eb468 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.044801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 416/853] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:54.046124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.077261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.078155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.078538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.078963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.079843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.080259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.081809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.101169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 438/854] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:07:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:54.104708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.184330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19880e-34b1-47e4-88f9-0948eec199f2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.199851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.226351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.234887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.235491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.235779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.236061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.236898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=0c95e2d016a84b5a804f4d18816a81ee) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.237211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.238694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.243438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f28f5-4b03-4887-9a0c-06a190c6f95f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.253502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 417/855] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/role_assignments?user.id=0c95e2d016a84b5a804f4d18816a81ee&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:54.284658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.286115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.286801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.287291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.300401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.302250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-99c816a3-c69c-490d-85ae-427ee80c083a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:54.323297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:54.324268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:54.325145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:54.325145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:54.325145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:54.325145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:54.327627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 439/856] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:54.333641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.395411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76693-da22-45a9-a99b-08396bd5481a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.395727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76693-da22-45a9-a99b-08396bd5481a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.395977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca76693-da22-45a9-a99b-08396bd5481a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.398623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 440/857] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:54.404350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.404675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.404916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.423812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:54.425048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:54.466246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9ca76-cf9c-4860-bd5b-20eb3eedc02b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.514444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.514758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 441/858] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:54.515604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.516088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.516536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.517277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.517641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.519032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.532402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 418/859] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:54.540149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.540349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.577849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.616882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.662316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a18d5-cfda-4621-b698-7f37d350b676 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.669373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491643b-9eae-4b41-998d-542bb2b85cda None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.699844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.700663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.700949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.701218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.701900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.703277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.716267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:54.717168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:54.718292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:54.719410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 419/860] 10.222.0.42 () {56 vars in 1132 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:54.720902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.721191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.721500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.724437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.731150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.732633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:07:54.752349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:54.753452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:54.754382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 442/861] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:54.762690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.858195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.867698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618887-df6b-4031-8ebe-3de613d1400f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.894797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d3d5-8c66-4161-aa1d-fb0396afb1f0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:54.912363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.913575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.913941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.914290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.915088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.915465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.917165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.922658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 420/862] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:54.923970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:54.934210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:54.946422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:54.946422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:54.947049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:54.947749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:54.949750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:54.950575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:54.954912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:54.980408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 443/863] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:54.992558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:55.062870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199c25-16ed-4ca0-bde1-2b470baf03e1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:55.105827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:55.107432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:55.107939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:55.108841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:55.109901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:55.110937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:55.112833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:55.121839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 421/864] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:55.132562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5ab9d-a252-4f53-8961-aae37ba97684 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:55.181829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:55.182894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:55.183378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:55.183868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:55.243252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:55.264235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:55.266184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:55.377753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 444/865] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:07:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:07:55.384132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:55.413779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:55.492904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480ed92-10c5-4065-972d-3635a234e75b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:55.530366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:55.531598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:55.532104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:55.532584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:55.545959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:55.547828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:55.566907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:55.567848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:55.568736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:55.568736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:55.568736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:55.568736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:55.571862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 422/866] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:55.577765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:55.668318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:55.687621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7fc9c-d949-4a85-af55-1fafb9437d34 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:55.731146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:55.731838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:55.732122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:55.732397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:55.733129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:55.733459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:55.734840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:55.748304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 445/867] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:55.753752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:55.852116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:55.858935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ceb15-b030-4eb4-a996-93174195d09b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:55.895228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:55.896051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:55.896338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:55.896624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:55.906077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:55.907556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:07:55.923650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:55.924573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:55.925483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 423/868] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:55.933067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:56.036312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ac64-1114-4a49-a498-9b3f6851df15 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:56.073876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:56.074602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.074892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.075172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.076023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:56.076335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:56.077882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42515590-70a3-4251-b5be-8fce08a3c1fc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:56.090901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 446/869] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:56.096383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:56.105409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697c06-710f-41fb-8934-5716e431126d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.105711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697c06-710f-41fb-8934-5716e431126d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.105998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697c06-710f-41fb-8934-5716e431126d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.107589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 447/870] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:56.112449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.112742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.113087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.119124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:56.131121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:56.135359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:56.203543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f20f7e-0416-4fcd-b5ed-9fb0ae85d8a2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:56.210575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 448/871] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:56.228070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:56.242081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:56.242740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.243023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.243299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.244131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ebdc88eed0ec4c19af2d34413c4f73e1) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:56.244446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:56.245927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a930211-071a-4d43-9ef4-88da23a6ab4b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:56.249690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:56.253525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 424/872] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:07:56 2022] GET /identity/v3/role_assignments?user.id=ebdc88eed0ec4c19af2d34413c4f73e1&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:56.348823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a33d8-df4a-42d6-865a-72a9b9c6f12f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:56.386709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:56.392493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:56.393234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.393854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.394153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.394909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:56.396719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd161f-27d0-4030-8e36-a6c816d2456b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:56.505514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 449/873] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:07:56 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:56.742189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45906c-0838-458e-9a7b-4176e99510a7 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.742510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45906c-0838-458e-9a7b-4176e99510a7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.742806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45906c-0838-458e-9a7b-4176e99510a7 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.744460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 425/874] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:56.750343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:56.750808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:56.750990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:56.772760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:56.774021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:56.858709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 450/875] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:56 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:56.877612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45906c-0838-458e-9a7b-4176e99510a7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:56.989057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45906c-0838-458e-9a7b-4176e99510a7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:57.026470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:57.027215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.027570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.027852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.029592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:57.031005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:57.044868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:57.045950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:07:57.046798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:57.050449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 426/876] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:07:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:57.052654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:57.161656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8145-9062-4d6a-8fca-7fdd6493efe5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:57.181752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:57.200407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:57.201081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.201391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.201677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.202398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:57.202721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:57.204089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ec2af-48a6-4cc9-92b6-a85d4422df90 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:57.214607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 451/877] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:07:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:57.216102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:57.286285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:57.336694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412ec3-512d-4654-b8ee-c46cc8f16980 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:57.375151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:57.375848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.376140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.376431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.377166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:57.377510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:57.378965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c57ef-5d52-4fa6-ac93-c14cf577cbcb None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:57.385874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 427/878] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:07:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:57.621588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4e5e-eae6-48a5-8c6b-68facfa25761 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.622563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4e5e-eae6-48a5-8c6b-68facfa25761 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.623308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f4e5e-eae6-48a5-8c6b-68facfa25761 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.626082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 452/879] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:57.635092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba1855-3c31-407f-bcd1-6630017aecc5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.635969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba1855-3c31-407f-bcd1-6630017aecc5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.636717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba1855-3c31-407f-bcd1-6630017aecc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.664799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a2ba1855-3c31-407f-bcd1-6630017aecc5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:57.667259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a2ba1855-3c31-407f-bcd1-6630017aecc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:57.784712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 428/880] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:57.815185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f4e5e-eae6-48a5-8c6b-68facfa25761 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:57.926680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f4e5e-eae6-48a5-8c6b-68facfa25761 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:57.976212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:57.977260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:57.977940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:57.978435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:57.979411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:57.980988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c2cc-5ce7-4535-ae36-10a1906c5020 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:58.083264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 453/881] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:07:57 2022] POST /identity/v3/services => generated 269 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:07:58.884099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f111-8050-45b4-b715-6d23302cc82a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:58.884682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f111-8050-45b4-b715-6d23302cc82a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:58.884682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0f111-8050-45b4-b715-6d23302cc82a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:58.886426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 429/882] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:58.892598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50603493-9e23-405a-9ab2-bfa13d270a69 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:58.896504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50603493-9e23-405a-9ab2-bfa13d270a69 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:58.896504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50603493-9e23-405a-9ab2-bfa13d270a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:58.913652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-50603493-9e23-405a-9ab2-bfa13d270a69 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:58.914891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-50603493-9e23-405a-9ab2-bfa13d270a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:58.966855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.005721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 454/883] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:07:59.029520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0f111-8050-45b4-b715-6d23302cc82a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.125606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a615f7-ff56-44f8-80ae-3f1b9786e200 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.125956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a615f7-ff56-44f8-80ae-3f1b9786e200 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.126176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a615f7-ff56-44f8-80ae-3f1b9786e200 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.127867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 455/884] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:07:59.133297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.133626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.133910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.137249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.156174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:07:59.157739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:07:59.214072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0f111-8050-45b4-b715-6d23302cc82a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.249938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 456/885] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:07:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:07:59.263881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.265010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.265537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.266023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.266884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.268487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.276223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:59.284464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:59.285813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:07:59.286664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:07:59.288619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 430/886] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:59.293896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.396321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.402247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7419b9-b117-4768-9cda-9bc859251cb1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.427810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cec57-48d7-40e3-8fdc-d4177721c762 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.443109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.445256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.445858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.446362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.453918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.461551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:07:59.470534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.471483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.471899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.472301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.473243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:59.473648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:07:59.474373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:59.475652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:07:59.476637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:59.477442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.479688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 457/887] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:07:59.482745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 431/888] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:59.487464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.493372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.617195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6dcf1-418f-410c-bc9d-faea02167027 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.628267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291899b-1016-4346-8e63-45ba0c043370 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.663310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.664088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.664710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.665456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:59.665769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.667367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.673442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.674617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.675046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.675486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.676331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:59.676791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.679051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.681652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 458/889] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:59.682410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.685485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 432/890] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:07:59.691636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.722088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.762040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:07:59.841029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b824ca-c17a-4570-ab18-fff0ba47613b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.841955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96803-19ba-453b-9225-fd16d461f0a0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:07:59.880503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.881229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.881840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.882313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.884217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] RBAC: Adding query filter params (interface=public, service_id=39df1280c505475e9403ef38f3745004, region_id=RegionOne) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:07:59.884769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.886379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.889968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:07:59.890949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:07:59.891865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:07:59.892293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:07:59.899792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 433/891] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/endpoints?service_id=39df1280c505475e9403ef38f3745004&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:07:59.900763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:07:59.906234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:07:59.909384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:07:59.937465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:07:59.938343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:07:59.939258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:07:59.939258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:07:59.939258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:07:59.939258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:07:59.941363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 459/892] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:07:59.953805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.036885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.045251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda8c19-0b98-4020-960c-b1103e35d88e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.065011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.073326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b3fd0-8a84-4fd6-9aa4-6934919694eb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.092163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.092857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.093507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.093794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.094547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.096110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.110706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 434/893] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/services => generated 1268 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:08:00.117533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.118290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.118589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.118879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.119633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:00.119956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.121512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412dc4-b44f-48c1-a504-57de18082ad8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.137138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 460/894] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:00.142740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.223052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7751cd-353b-4c17-9127-65eaac02fb8a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.223233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7751cd-353b-4c17-9127-65eaac02fb8a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.223523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7751cd-353b-4c17-9127-65eaac02fb8a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.226086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 461/895] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:00.233081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.233770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.234255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.253209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:00.255038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:00.269947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac70ce-a96a-42d7-901a-63dccfdb1e5e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.314097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.315512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.315973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.316398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.326324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.328093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.339125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 462/896] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:00.345917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:00.347089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:00.348249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:00.349401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 435/897] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:08:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:00.355195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.367511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.422316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.434983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.477229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4e13d-b3d3-42cf-ac70-ffbfdef10b28 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.491519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1547fb-b2ca-4d4d-9ba8-9c161be8e95e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.516246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.517043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.517334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.517640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.518484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:00.518799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.520258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be31ea-ad6e-4790-afa9-425632a8f3a0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.533858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.535084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.535650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.536723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.538548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.540145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.540799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 463/898] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:08:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:00.542250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dee86-1334-4997-9015-7379a2bb0103 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.542551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dee86-1334-4997-9015-7379a2bb0103 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.542833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dee86-1334-4997-9015-7379a2bb0103 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.544228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 464/899] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:00.550295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.550589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.550873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.567860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:00.569171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:00.622942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 436/900] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:08:00 2022] POST /identity/v3/services => generated 241 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:00.624797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.687604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 465/901] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:00.726898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:00.762700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.768017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4456b45-0749-4937-bb8f-f8e976e2ed9f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.816934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.816934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.816934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.816934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.836096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:00.913807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.915287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.919296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce26010-45af-4a56-a6a2-7967053bfc16 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:00.962612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:00.963433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:00.963716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:00.963997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:00.964661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:00.966074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:00.978540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:00.980777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:00.983733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 466/902] 10.222.0.42 () {56 vars in 1132 bytes} [Fri May 27 16:08:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:00.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.036384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.049442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 437/903] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:00 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:01.054322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.129519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2fabc-6dc1-48c4-b93e-f21c5c147697 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.141257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.166661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.167340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.167625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.167904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.168628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:01.168938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.170266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15d7bd-50ea-456b-8534-2ecb3cbf4219 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.170411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.175983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 467/904] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:01.182296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.213213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.214104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.214392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.214673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.225830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.227488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:01.247649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:01.248614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:01.249574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:01.249574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:01.249574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:01.249574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:01.253410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 438/905] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:01.255712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.315282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.321230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ab456-5733-4759-97dd-c47bf1aa93c2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.359736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.360505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.360695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.360980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.361742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:01.362076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.363488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.373387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 468/906] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:01.376151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.388814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e1fe9-f803-4f71-b7f9-5adf9486f512 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.434204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.435391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.436425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.436816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.437200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.438066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:01.438479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.440442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.461575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 439/907] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:01.468068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.522128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5dd2-116d-4c97-b9d2-860119b58c02 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.561966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.562997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.563596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.564092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.565217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] RBAC: Adding query filter params (interface=public, service_id=4632d61c913841dd88b13f65c183d934, region_id=RegionOne) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:01.565837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.567551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.579174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 469/908] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/endpoints?service_id=4632d61c913841dd88b13f65c183d934&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:01.580720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.590485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5d0e1-38e4-4cb7-b9e7-41ec9bd3576d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.603703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.631991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.632815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.633104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.633408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.641891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.643429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:01.664366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:01.665391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:01.666308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:01.669481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 440/909] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:01.671810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.703220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7bed2-e33a-4545-98ae-a638f2ae64ec None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.705272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:01.743732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.744925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.745450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.746011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.746960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.748751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.757523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 470/910] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:08:01.786976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c7810-2ed0-4d3e-a638-2663070441ad None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:01.830817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:01.831438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:01.831724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:01.832006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:01.832812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:01.833127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:01.835276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1da24-050e-4b33-8680-35e6b3661327 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:01.856352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 441/911] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:01.858826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:01.986243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146921d4-1ec5-43c5-a815-a72ee1d8c6fb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:02.025040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:02.026132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:02.026659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:02.027139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:02.028185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] RBAC: Adding query filter params (role.id=f206887ec47b42c0901d2d75ea1cb23d, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ebdc88eed0ec4c19af2d34413c4f73e1) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:02.028727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:02.030450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960d10-c053-4f12-b6be-cc6362a2bdd9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:02.037757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 471/912] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:01 2022] GET /identity/v3/role_assignments?user.id=ebdc88eed0ec4c19af2d34413c4f73e1&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=f206887ec47b42c0901d2d75ea1cb23d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:02.622277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70def0-95c0-4e55-8bbd-bcdf7a5c26c1 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:02.622803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70def0-95c0-4e55-8bbd-bcdf7a5c26c1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:02.622803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70def0-95c0-4e55-8bbd-bcdf7a5c26c1 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:02.624444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 442/913] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:02.630182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:02.630479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:02.630767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:02.649259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:02.651224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:02.679088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:02.725934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 472/914] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:02.748508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70def0-95c0-4e55-8bbd-bcdf7a5c26c1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:02.780888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:02.861889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70def0-95c0-4e55-8bbd-bcdf7a5c26c1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:02.901256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:02.902388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:02.902824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:02.903252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:02.904081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:02.905657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:02.919586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:02.920895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 16:08:02.921980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:02.924800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 443/915] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:08:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:02.931676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.065084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e164-cd66-40d3-b48e-af3be76c557b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.102596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.103284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.103574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.103859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.104584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:03.104898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.106315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65afb7-3275-4d14-a1a0-685b085978da None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.111873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 473/916] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:08:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:03.117472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.137044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666174b-9a02-4976-ba43-75dded09d8f7 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.137353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666174b-9a02-4976-ba43-75dded09d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.137646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666174b-9a02-4976-ba43-75dded09d8f7 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.139273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 474/917] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:03.144439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.144726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.145008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.164558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:03.165786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:03.201848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:03.228060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:03.235061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3bb81-204b-4bd0-bb49-6874921301a6 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.238894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 475/918] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:03.261817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.271700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.272617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.272904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.273179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.274056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:03.274365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.275751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.281242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 444/919] 10.222.0.42 () {56 vars in 1114 bytes} [Fri May 27 16:08:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:03.288805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.385570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fedaf-3941-43a5-a3cd-f548bde4bf1a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.422203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.422927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.423213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.423498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.424665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.426206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.429087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61070e31-ad47-4b4d-a4eb-e4e8748a71b5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:03.439704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:03.440777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:03.441674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 476/920] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:03.449096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.472811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:03.481911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.489848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.489848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.489848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.489848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.489848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.559119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dbd66-6868-4671-b69f-0444f1d33675 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.577387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 445/921] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:03.619796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.620540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.620828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.621104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.621907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:03.622271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.623739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.630036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 477/922] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:03.636194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.657751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:03.765944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ca08a-2d73-44a6-9ccb-96468d3d55fc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.806399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.807514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:03.808248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:03.808775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:03.809730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:03.810237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:03.811886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:03.819277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 446/923] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:03.826075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:03.919171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:03.956014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff4149-0c6d-48fa-ac86-444847439924 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:03.998464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:03.999569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.000056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.000545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.001708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] RBAC: Adding query filter params (interface=public, service_id=db20d0b2d65e45c487dd71741f4ec44d, region_id=RegionOne) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:04.002295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:04.003931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4472182-c506-455e-87e9-9a18548a2fbe None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:04.011196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 478/924] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:08:03 2022] GET /identity/v3/endpoints?service_id=db20d0b2d65e45c487dd71741f4ec44d&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:04.017056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.108125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:04.133822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83ed-a11a-4462-97f0-b67cf594e5bf None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:04.175229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:04.175905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.176190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.176472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.177157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:04.179143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:04.182404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0461b-37df-4ad8-a31a-13a971f44b5f None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.182886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0461b-37df-4ad8-a31a-13a971f44b5f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.182969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0461b-37df-4ad8-a31a-13a971f44b5f None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.184612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 479/925] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:04.186108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 447/926] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:08:04.189989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eb72-8d9d-4271-9a25-b55676a8b36b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.190280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eb72-8d9d-4271-9a25-b55676a8b36b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.190564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9eb72-8d9d-4271-9a25-b55676a8b36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.210010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-40f9eb72-8d9d-4271-9a25-b55676a8b36b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:04.211319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-40f9eb72-8d9d-4271-9a25-b55676a8b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:04.289404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 480/927] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:04.314453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.471262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb1e8-5f42-48ee-8643-372c87d89eb5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:04.529722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:04.530577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.530868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.531148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.531464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f29c6-bb45-4cc1-82a6-01665f82a93f None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.531807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:04.531988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f29c6-bb45-4cc1-82a6-01665f82a93f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.532491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f29c6-bb45-4cc1-82a6-01665f82a93f None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.533194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:04.535490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 481/928] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:04.544610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.545059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.545675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:04.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:04.548510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:04.549331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 448/929] 10.222.0.42 () {56 vars in 1132 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:04.561380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.571807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:04.576776 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:04.578092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:04.669611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 482/930] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:04.687931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a721571-f9c8-4de5-bd12-c8e6d69df098 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:04.688932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.692671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:04.732646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:04.733520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.733907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.734287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.735094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:04.735503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:04.737210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:04.747655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 449/931] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:04.748692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.817897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a25fa6-0c54-4c06-a901-682c82359de8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:04.830212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:04.864448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:04.864666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e334-7cda-42f5-85fa-c5d807a24cbb None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:04.865879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.866177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.866456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.881295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:04.892141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:04.893598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:04.894798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 483/932] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:04.901008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.911803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:04.912619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:04.913009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:04.913359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:04.914369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:04.914704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:04.916326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:04.926679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 450/933] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:04.927587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:04.998879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:05.039787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925e68b-6d14-4e9b-824b-bad93cb720fa None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.058275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2524a64-b33a-40cf-99c0-5fc918ec8cb1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.079740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.080488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.080970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.081387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.082352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:05.082766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.084777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.094331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 484/934] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:08:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:05.099917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:05.106496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.107162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.107449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.107732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.108480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.109915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.113391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:05.210257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5d06d-c6db-46ab-9e2d-aa17e7f0f2ee None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.234167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:05.246121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.246952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.248592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.248592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.258240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.259699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:05.277707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:05.283770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:05.284920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:05.284920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:05.284920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:05.284920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:05.284920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 485/935] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:05.288831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:05.289992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 451/936] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:05.411109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45c1ea-7126-4287-b812-812ec6d8ddf1 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.457972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.458699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.459312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.459451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.460247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:05.460564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.462034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.476840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 486/937] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:05.482401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:05.592678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823aca-9483-433d-ac4e-3701e6e0af73 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.627933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.628749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.629036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.629313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.637607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.639129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:05.654820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:05.655802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:05.656720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 452/938] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:08:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:05.662334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:05.695634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:05.767843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11fe2-81ad-4819-af85-f47bf2d4f573 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.804903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.805616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.805905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.806188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:05.807036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:05.807347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:05.808786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:05.827675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 487/939] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:08:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:05.833786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:05.901727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:05.954668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e5b42-ec65-431f-8f63-7a9d7d5211b9 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:05.992431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:05.993240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:05.993560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:05.993838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.005411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.007712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3e2a355e-5abb-456b-aab7-ec5147568a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:06.025885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:06.026807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:06.027691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:06.027691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:06.027691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:06.027691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.027691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 453/940] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:06.033862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.043531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2c35-9891-4723-b338-34ac2416abf3 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.043818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2c35-9891-4723-b338-34ac2416abf3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.044095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f2c35-9891-4723-b338-34ac2416abf3 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.045923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 454/941] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:06.050510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.050800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.051174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.069371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:06.070524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:06.146184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 455/942] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:06.159979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.171505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8049a5c-3381-412b-8cde-bf449e89705c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.174436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.211643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.225136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.227028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.227466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.227902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.228944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:06.229421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.231630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.248355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 488/943] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:06.253774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.295655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe74f0-bdb8-4fab-a8f8-9b49cb1922dc None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.339899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.340592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.340878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.341156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.341921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.343443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.344891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING keystone.server.flask.common [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 16:08:06.388642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7383-df25-4901-8f05-7c7eb43681d2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.394328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.sql.core [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 16:08:06.394328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 16:08:06.394328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [parameters: {'id': 'bc46d93c2ba645ecaa7dbc45d7eb6a7d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 16:08:06.394328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc46d93c2ba645ecaa7dbc45d7eb6a7d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc46d93c2ba645ecaa7dbc45d7eb6a7d. May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application conn.query(q) May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:08:06.419994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result.read() May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 16:08:06.420851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application next(self.gen) May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application yield resource May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application next(self.gen) May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.session.rollback() May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.force_reraise() May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise self.value May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application session.commit() May 27 16:08:06.422299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._prepare_impl() May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.session.flush() May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._flush(objects) May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application compat.raise_( May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application flush_context.execute() May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 16:08:06.423191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rec.execute(self) May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 16:08:06.424077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application conn.query(q) May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result.read() May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:08:06.424866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [parameters: {'id': 'bc46d93c2ba645ecaa7dbc45d7eb6a7d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.425772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:06.426548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc46d93c2ba645ecaa7dbc45d7eb6a7d. May 27 16:08:06.427436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.430127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 456/944] 10.222.0.42 () {60 vars in 1140 bytes} [Fri May 27 16:08:06 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:06.431265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.435227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.436037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.436317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.436591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.444801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.446540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:06.464337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:06.465244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:06.466160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 489/945] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:06.472121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.488154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.555443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.570337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b7490-7346-4586-a9d7-5fde5ffa6611 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.595356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9bcca-d551-4882-ae81-4fcb9b44ec47 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.605486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.606667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.606941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.607501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.614270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.616526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8b82a50d-1c7e-4b07-af12-b5f403523954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:06.631766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:06.632699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:06.633604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 457/946] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:06.634615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc005-ebdc-4013-986e-089cf10821c3 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.634685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc005-ebdc-4013-986e-089cf10821c3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.634925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741cc005-ebdc-4013-986e-089cf10821c3 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.636202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.636861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.637144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.637702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.638368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:06.638678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.640117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ea843-6cba-479c-9cb9-494ca7fb224f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.646948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 458/947] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:06.648634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cc005-ebdc-4013-986e-089cf10821c3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.653481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 490/948] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:06.655244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.655244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.655438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.672827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:06.674234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:06.741172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.746314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 491/949] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:06.747099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.777118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741cc005-ebdc-4013-986e-089cf10821c3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.813655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.814689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.815141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.815425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.816181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:06.816491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.817954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.845368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 459/950] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:06.845368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:06.856366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:06.868138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5c398-3824-4fb4-bbf8-eda6006ce418 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:06.919535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:06.920892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:06.921254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:06.922324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ebdc88eed0ec4c19af2d34413c4f73e1/roles/f206887ec47b42c0901d2d75ea1cb23d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:06.926158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=ebdc88eed0ec4c19af2d34413c4f73e1, role_id=f206887ec47b42c0901d2d75ea1cb23d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:06.931803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:06.975806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e6e61-e7dd-4473-80b8-f9ba75b5c606 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:07.016706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:07.017577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.017873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.018160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.018830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:07.020240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:07.021730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 492/951] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:08:06 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ebdc88eed0ec4c19af2d34413c4f73e1/roles/f206887ec47b42c0901d2d75ea1cb23d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:07.034603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:07.035863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.037996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 460/952] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:07.043010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:07.119755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:07.170751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0dd41-6573-4cc7-a5c1-5a691ea719f9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:07.211526 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:07.212652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.213173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.213693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.214621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:07.215171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:07.216765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:07.223329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 493/953] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:07.229042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:07.308851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:07.339891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f3ec3-1f71-4020-a769-ec3a5373dfc3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:07.375782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:07.376449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.376732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.377005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.377744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:07.378052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:07.379425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea31a24-8988-458b-9858-a60786633f2f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:07.385070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 461/954] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:07.390678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:07.499728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d6d7-801c-44ec-bd78-78a4b3986291 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:07.524781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:07.535447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:07.536104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.536387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.536663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.538280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:07.539789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53c250-1694-4204-9d75-004b0ddb3da6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:07.624539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 494/955] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:07.670172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd10535-8f5a-4178-b382-c42b5ea9f19a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.670465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd10535-8f5a-4178-b382-c42b5ea9f19a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.670747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd10535-8f5a-4178-b382-c42b5ea9f19a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.672429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 462/956] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:07.679020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.679671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.680154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.700140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:07.701789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:07.790699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 495/957] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:07.812783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd10535-8f5a-4178-b382-c42b5ea9f19a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:07.923645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd10535-8f5a-4178-b382-c42b5ea9f19a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:07.961743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:07.962538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:07.962821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:07.963099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:07.963752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:07.965115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:07.967904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:07.978206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:07.980035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:07.980892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 463/958] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:07.985526 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:08.090340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdc33c-b58a-4e44-8d08-f924019eaa10 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:08.098025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:08.125855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:08.126525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:08.126814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:08.127094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:08.127804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:08.128118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:08.129534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30afa931-c0c1-40fb-8e40-ba070318102e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:08.134836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 496/959] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:08.140516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:08.250518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573940a1-8ee0-404d-ab30-f9e115100d29 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:08.286452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:08.287788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:08.288086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:08.288367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:08.289080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:08.289407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:08.290795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25b11d-c584-40df-a80b-8fd1212f5a67 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:08.296028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 464/960] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:09.331984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ec43-deb2-4aee-98fa-e91365fdde03 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.332474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ec43-deb2-4aee-98fa-e91365fdde03 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.332474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ec43-deb2-4aee-98fa-e91365fdde03 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.334319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 497/961] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:09.340273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.341173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.341704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.346287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce843-a2a6-431a-89a9-9edb46e4d9f7 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.346812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce843-a2a6-431a-89a9-9edb46e4d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.347309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce843-a2a6-431a-89a9-9edb46e4d9f7 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.348847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 498/962] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:09.354985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.355525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.356023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.359756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:09.361516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:09.373989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:09.375484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:09.437462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:09.447447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 465/963] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:09.450163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 499/964] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:09.473108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.484966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.596644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:09.601979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bbe6b-6a57-416b-a090-b8749bc22b34 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:09.612679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2433c-8e6d-4daa-aa20-a50ea51ba7f3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:09.639460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:09.639665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:09.640133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.640586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.640944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.641196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.641432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.641919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.642054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:09.643678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:09.654563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:09.664173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:09.665134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:09.666356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:09.667040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 500/965] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:08:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:09.672820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.691488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.sql.core [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 16:08:09.691488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 16:08:09.691488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [parameters: {'user_id': '33585d15040d4f44a723b9f27ca8e8fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 16:08:09.691488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application conn.query(q) May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result.read() May 27 16:08:09.699704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:08:09.700575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application next(self.gen) May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application yield resource May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application next(self.gen) May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.session.rollback() May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.force_reraise() May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise self.value May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application session.commit() May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 16:08:09.701530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._prepare_impl() May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.session.flush() May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._flush(objects) May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application compat.raise_( May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application flush_context.execute() May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rec.execute(self) May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 16:08:09.702441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 16:08:09.703392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result = self._query(query) May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application conn.query(q) May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application result.read() May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 16:08:09.704227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [parameters: {'user_id': '33585d15040d4f44a723b9f27ca8e8fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:09.705046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 16:08:09.705861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 16:08:09.706768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.714810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 466/966] 10.222.0.42 () {60 vars in 1131 bytes} [Fri May 27 16:08:09 2022] POST /identity/v3/users => generated 160 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:09.716156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.777901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:09.819166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453f009-297b-41ce-b289-061af85fe073 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:09.831157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c1a4-e868-483d-b0e8-b72f641e4552 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:09.865135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:09.870487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.870487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.870487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.870487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:09.870487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:09.870835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:09.870933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:09.871011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:09.871011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:09.871011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:09.879916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:09.882464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:09.884963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 501/967] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:08:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:09.885931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:09.900402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:09.901415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 16:08:09.902341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:09.909420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 467/968] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:09.911114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:09.913983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.005257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd9df3-3dfa-447c-b754-7dd1646b6422 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.031090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632701dc-bc3b-42e9-a857-aa034dc2a9c0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.042295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.043091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.043370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.043643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.050489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.053943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.055418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.070595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.071592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.072191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.072672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.073684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:10.074165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-3d37b582-481a-49db-86c1-1396ee1af21c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:10.075196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:10.076053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:10.077241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:10.077241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:10.077241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:10.077241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:10.077241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 502/969] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:10.077744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331862-2f40-400e-8bcc-641585559278 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.077928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331862-2f40-400e-8bcc-641585559278 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.078204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97331862-2f40-400e-8bcc-641585559278 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.078908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272ba5d-71eb-40e7-b5a2-396f1a2e6198 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.083743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 503/970] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:10.083743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97331862-2f40-400e-8bcc-641585559278 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.124486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 468/971] 10.222.0.42 () {56 vars in 1101 bytes} [Fri May 27 16:08:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:10.127030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.127565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.128378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.154808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:10.159730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:10.169407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.204982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97331862-2f40-400e-8bcc-641585559278 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.249375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 469/972] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:10.257248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.257977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.258263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.258542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.259282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:10.259598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.261014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.271372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.274982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 504/973] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:10.280567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.400254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca88203-edf5-4fa3-9e0e-cba25fce9b4e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.423245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953f19-8398-4465-b222-5802b0d06df1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.426388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.450384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.451601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.451905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.452172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.456989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.466552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.468009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.468887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.469733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.470016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.470294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.470958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.475215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:10.484613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:10.485853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:10.486867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 505/974] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:10.489051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:10.490306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:10.491180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:10.492715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 470/975] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:10.494368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.497364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.602855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e03e-8b49-4c9f-976f-2e45bcf975fb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.612227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82b4aa-a8e8-446c-bfd6-1756fdf9f336 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.636246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.636913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.637208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.637508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.638348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:10.638658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.640099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eda1e-4a23-4b65-9a9f-1a03f8a93cb6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.645297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.645993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.646991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.646991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.647979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:10.648283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.649651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c785edf-3ac0-4aea-818d-30464e6a9503 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.654429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 471/976] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:10.655650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c39f-e6c5-4752-946a-bec11c8efe73 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.655994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c39f-e6c5-4752-946a-bec11c8efe73 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.656277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c39f-e6c5-4752-946a-bec11c8efe73 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.660509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 472/977] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:10.660509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c39f-e6c5-4752-946a-bec11c8efe73 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.663640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 506/978] 10.222.0.42 () {56 vars in 1149 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:10.669367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.669793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.670080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.686651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:10.687862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:10.691038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.735592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.756313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 507/979] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:10.757144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.763525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c39f-e6c5-4752-946a-bec11c8efe73 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.798852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.799518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.799807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.800090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.800788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:10.801097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.803207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.808523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 473/980] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:10.814101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:10.862595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a2e41-4685-4aa8-9c1c-8274d6a67bbf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.898087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.898840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.899127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.899410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.916927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.922481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69dd9c-bea6-450e-9af6-f018964d48c2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:10.951811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:10.956653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:10.957328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:10.957638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:10.957919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:10.959170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.960768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:10.976580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:10.978091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.017441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 474/981] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:08:10 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:11.018296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.106168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 508/982] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:10 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:11.112303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.150373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93007714-3531-4f01-a949-ced8e3128ae0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.191417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.192643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.193234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.193732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.194941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] RBAC: Adding query filter params (interface=admin, service_id=db20d0b2d65e45c487dd71741f4ec44d, region_id=RegionOne) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:11.195984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.197911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.207972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 475/983] 10.222.0.42 () {56 vars in 1249 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/endpoints?service_id=db20d0b2d65e45c487dd71741f4ec44d&interface=admin®ion_id=RegionOne => generated 193 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:11.212402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:11.214735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.235001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99dd54-4322-467f-a10e-a4ce5593309f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.237700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:11.273415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.274480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.274770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.275046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.287650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.289108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:11.307203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:11.308142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:11.309034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:11.309034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:11.309034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:11.309034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:11.309034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 509/984] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:11.313876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.341270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54b7c8-3271-4fb6-854f-0b9b272760bc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.377618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.378284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.378567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.378847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.379580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.381907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.387970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 476/985] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:08:11.432493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5895050-efd4-4c5f-b394-763598b64280 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.471449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.472039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.472322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.472596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.473328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:11.473671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.475962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.477744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:11.493386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 510/986] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:11.496517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.590799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:11.613579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7b41e-306d-40bf-a421-a47d31281eb7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.650520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.651774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.652379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.653201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.660435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.663141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:11.676610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:11.677799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:11.678695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 477/987] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:11.684256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.789581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03b2e0-3cc7-4530-b751-37857bcf1271 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:11.824625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:11.825292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:11.825604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:11.825889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:11.826677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:11.826992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:11.828437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b6ee-f41a-429d-9de6-2f6c5f4a3b01 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:11.847586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 511/988] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:11.848752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:11.965757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08854a1e-70f8-4606-80e1-b40e7d5d3f50 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:12.002448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:12.003108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:12.003395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:12.003673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:12.004503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] RBAC: Adding query filter params (role.id=f206887ec47b42c0901d2d75ea1cb23d, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ebdc88eed0ec4c19af2d34413c4f73e1) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:12.004824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:12.007797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeaecdc-c286-4669-9f1e-623c5d7f139f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:12.022102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 478/989] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:11 2022] GET /identity/v3/role_assignments?user.id=ebdc88eed0ec4c19af2d34413c4f73e1&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=f206887ec47b42c0901d2d75ea1cb23d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:13.307605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2954-ffaf-46f7-b272-44e821d240f5 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.308119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2954-ffaf-46f7-b272-44e821d240f5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.308119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf2954-ffaf-46f7-b272-44e821d240f5 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.309607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 512/990] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:13.314766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.315139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.315345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.321796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:13.333923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:13.335659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:13.403326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 479/991] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:13.427071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf2954-ffaf-46f7-b272-44e821d240f5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.489049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:13.536323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf2954-ffaf-46f7-b272-44e821d240f5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:13.570626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:13.571347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.571634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.571911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.572571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:13.573992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-e843f743-fce6-442d-92eb-df7e01c89ba1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:13.585932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:13.587014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:13.587860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.589424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 513/992] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:13.593560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.618693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc333ce-f77b-461a-9392-caaa6e8ca276 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.618991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc333ce-f77b-461a-9392-caaa6e8ca276 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.619303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc333ce-f77b-461a-9392-caaa6e8ca276 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.620741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 514/993] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:13.625410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.625703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.625983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.643683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:13.644883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:13.703106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a27bd-951f-43b3-a12e-1a7e2145e258 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:13.718843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 515/994] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:13.739053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:13.739993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.740390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.740654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.741398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:13.741598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.741716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:13.743090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:13.744258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:13.753888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 480/995] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:13.754929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.792073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:13.848408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ed67f-dd72-4235-98e3-d83757f3283d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:13.865008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef161f-1bad-420a-9ebd-c2e7901d256c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:13.886778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:13.887718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.888025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.888293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.888954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:13.890482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:13.899452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:13.900120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:13.900404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:13.900679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:13.901400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:13.901710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:13.903072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:13.903226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:13.904145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:13.904980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 516/996] 10.222.0.42 () {56 vars in 1126 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:13.915707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 481/997] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:13.915893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.916818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:13.987022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.027997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510c0c-8ba1-4709-a32f-196fb6587768 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.030454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5330c-2df4-4e13-87fa-530acc6f0ead None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.045090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.065247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.065949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.066231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.066505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.067302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] RBAC: Adding query filter params (interface=public, service_id=b39b66a9fb834e5684a239c3d61b248d, region_id=RegionOne) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:14.067616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.069794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.080664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 482/998] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/endpoints?service_id=b39b66a9fb834e5684a239c3d61b248d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:14.081989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.082185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.082878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.083169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.083526 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.084414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:14.084680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.086193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.091843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 517/999] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:14.097830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.197684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf104ebc-8649-428b-96d2-9cb86453cd02 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.203389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d9c45-8492-447c-9e34-ce2d237d665c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.222757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.234588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.235584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.235584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.235861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.236469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.237906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.244030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.251862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efb992-ca2d-48f5-8a42-4daae6e1c6e7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.253120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 483/1000] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:08:14 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:08:14.260279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 518/1001] 10.222.0.42 () {56 vars in 1122 bytes} [Fri May 27 16:08:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:14.266202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.272032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4df1-233e-4ee3-8608-5eeec310aa1c None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.272407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4df1-233e-4ee3-8608-5eeec310aa1c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.272612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4df1-233e-4ee3-8608-5eeec310aa1c None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.274272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 519/1002] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:14.279865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.280165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.280451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.299278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:14.300517 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:14.342524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.382957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 520/1003] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:14.393199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc2ceb-3a56-434a-bced-85a3dd8b8238 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.404844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.434115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.434777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.435063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.435340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.436094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.437509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.513172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af52ce8-9584-49ed-907b-a72ab000580b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.520727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 484/1004] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:14.571394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.572259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.572551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.572850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.573581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.575032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:14.589478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:14.590649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:14.591536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 521/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Fri May 27 16:08:14 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:14.596962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.655047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.715800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c2bdc-eae9-443d-bafd-33e30083c8b3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.751751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.752422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.752706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.752982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.753721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] RBAC: Adding query filter params (name=image) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:14.754031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.755966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192f70-f7a6-4975-8085-0df56e50b08b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.761602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 485/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:14.767097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:14.799832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:14.871945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30068e98-5c06-402c-ad2d-058b9d7a05ae None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:14.907068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:14.907739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:14.908026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:14.908308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:14.909270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:14.909608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:14.910989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa98ce-30d5-4021-af2c-af4bf1745ee2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:14.921438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 522/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:16.495048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa264-3955-4a2b-9bec-cedd8dfd8207 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.495567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa264-3955-4a2b-9bec-cedd8dfd8207 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.495676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa264-3955-4a2b-9bec-cedd8dfd8207 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.497137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 486/1008] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:16.502065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f11d-44de-4b6b-b433-ab0eabfa1773 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.502436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f11d-44de-4b6b-b433-ab0eabfa1773 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.502636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8f11d-44de-4b6b-b433-ab0eabfa1773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.520209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b4a8f11d-44de-4b6b-b433-ab0eabfa1773 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:16.521627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b4a8f11d-44de-4b6b-b433-ab0eabfa1773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:16.594731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 523/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:16.619477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa264-3955-4a2b-9bec-cedd8dfd8207 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:16.728926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa264-3955-4a2b-9bec-cedd8dfd8207 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:16.743160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:16.763658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:16.764378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.764664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.764941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.765615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:16.766994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:16.774815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3f9e-f8d8-48e1-a279-a68542e5fafd None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.775109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3f9e-f8d8-48e1-a279-a68542e5fafd None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.775654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d3f9e-f8d8-48e1-a279-a68542e5fafd None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.777388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 524/1010] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:16.780416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:16.781420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 16:08:16.782859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:16.784553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 487/1011] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:16.785066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.785375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.785661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.788076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:16.803607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:16.804825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:16.823150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:16.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 525/1012] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:16.915778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf79f2-be83-4ed8-b92b-1dce6bd40d79 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:16.916282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:16.961322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:16.962082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:16.962383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:16.962897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:16.963360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:16.963746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:16.965082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:16.971045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 488/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:16.976683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:17.026117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:17.039637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a16a1f-d704-4571-8f3d-36c32189a4df None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:17.077844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:17.078985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.079481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.079855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.081022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:17.082660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:17.085374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 526/1014] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:17.087432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e1fa7-dc00-4605-aed1-38db23f0d9d1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:17.091688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:17.123296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:17.125556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:17.126219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.126508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.126790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.127512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:17.127803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:17.129169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:17.139106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 489/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Fri May 27 16:08:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:17.140513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:17.200729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea7fb-fc7b-47f2-a5ff-7826a190aeb9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:17.234955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:17.235618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.235903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.236185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.236893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:17.238365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be9707-0b6b-40e3-b976-88a04d9af0c7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:17.243059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b2558-e89e-44e1-93c7-4b72a45e0c06 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:17.249876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:17.277057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:17.277688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.277975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.278252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.278979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:17.280341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:17.303366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 527/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:08:17 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:17.304824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecede3-c8a2-4a3f-a290-41ce4e0a0de0 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.305126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecede3-c8a2-4a3f-a290-41ce4e0a0de0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.305544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecede3-c8a2-4a3f-a290-41ce4e0a0de0 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.306836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 528/1017] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:17.312766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395af5-825b-47b4-b0d8-cdbeede22bfc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.313058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395af5-825b-47b4-b0d8-cdbeede22bfc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.314172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70395af5-825b-47b4-b0d8-cdbeede22bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.332040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-70395af5-825b-47b4-b0d8-cdbeede22bfc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:17.333914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-70395af5-825b-47b4-b0d8-cdbeede22bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:17.359963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 490/1018] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:17.418552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:17.425774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 529/1019] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:17 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:17.452208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:17.576610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7afff-dab4-4603-be42-31975dd48c81 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:17.611798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:17.612468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:17.612753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:17.613030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:17.613800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:17.615808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3392f-914e-4449-887c-ec49b15319db None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:17.686862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 491/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:08:17 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:19.596596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b81f77-278b-4a23-b9da-94062db98296 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.597117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b81f77-278b-4a23-b9da-94062db98296 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.597225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b81f77-278b-4a23-b9da-94062db98296 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.598596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 530/1021] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:19.603937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a22bb-13bf-4cf2-8afc-b9a3270e2a4d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.604329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a22bb-13bf-4cf2-8afc-b9a3270e2a4d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.604510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a22bb-13bf-4cf2-8afc-b9a3270e2a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.621210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e26a22bb-13bf-4cf2-8afc-b9a3270e2a4d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:19.622530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e26a22bb-13bf-4cf2-8afc-b9a3270e2a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:19.639709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:19.696951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 492/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:19.703115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa072d-23d4-414e-8fa1-62a17085790a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.703417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa072d-23d4-414e-8fa1-62a17085790a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.703699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa072d-23d4-414e-8fa1-62a17085790a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.705175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 531/1023] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:19.710498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.710794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.711078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.724176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa072d-23d4-414e-8fa1-62a17085790a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:19.728902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:19.730196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:19.801920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 493/1024] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:19.826026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:19.846070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa072d-23d4-414e-8fa1-62a17085790a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:19.891262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:19.899674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 532/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:19.903303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:19.919847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:19.944706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ebb32-1b12-4053-83b7-4aee81f91fb0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:19.980778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:19.981618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:19.982074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:19.982358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:19.983025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:19.994268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-31e12404-b259-4225-a8cd-3f2ee696513e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:20.002498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:20.003427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:20.004277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 494/1026] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:08:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:20.005462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b934-44d2-42ed-8c0d-be5651851ec4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.005569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b934-44d2-42ed-8c0d-be5651851ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.010056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011b934-44d2-42ed-8c0d-be5651851ec4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.010056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 495/1027] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:20.010645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011b934-44d2-42ed-8c0d-be5651851ec4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.018048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bae1f0-cd1b-47ba-aa81-38ecbf599658 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.055270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.055947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.056235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.056515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.057224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.058713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.114731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011b934-44d2-42ed-8c0d-be5651851ec4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.149893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.150492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.150780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.151061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.151809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.152141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.153533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dafe2-4302-4c74-97bc-77285ec28a42 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.159192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.163623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 496/1028] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:20.177592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.177592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.177823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 533/1029] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:08:19 2022] POST /identity/v3/users => generated 285 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:20.177925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.178056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.193912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:20.195333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:20.280598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 497/1030] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:20.287187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bdb3-086b-448e-82b6-72dae1996d6f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.306836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.315109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.329021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.330183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.330622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.331066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.343794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.345697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:20.365432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:20.366455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:20.367492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:20.367492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:20.367492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:20.367492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:20.367492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 534/1031] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:20.372121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.436061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cd8f7-d89a-4c9f-971d-fd009c3a4db0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.467422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.473313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.474438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.474724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.475046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] PATH_INFO: `/v3/services/image` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.475699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.477071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.485386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2eea-74b1-4f94-9d02-d3fc99670466 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:20.489147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:20.490323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:20.491325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:20.493501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 498/1032] 10.222.0.42 () {56 vars in 1105 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:20.497254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.527753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.528420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.528706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.528990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.529735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.530044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.531420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.543823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 535/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:20.549024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.588191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.614020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93814f6b-1221-4eee-9cb4-527446d1027f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.654268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.654983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.655266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.655573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.656280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] RBAC: Adding query filter params (name=image) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.656587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.657981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.659827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec058c-01ba-45d2-94ba-7d4fbf8cec99 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.664908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 499/1034] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:20.670567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.697753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.698567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.698857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.699134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.706905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.709115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:20.725212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:20.730199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:20.731294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 536/1035] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:20.731957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.742275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.782305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e636-0ded-473b-86ef-fe342ded606b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.820355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.821134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.821557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.821734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.822448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] RBAC: Adding query filter params (type=image) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.822760 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.824157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.829438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 500/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:20.835775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.836327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.842142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08b891-c981-40e9-a882-7bac3d3982e0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.878525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.879206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.879495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.879874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.881106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.881456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.883443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.902278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 537/1037] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:20.907675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:20.949010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906f3f4-d004-4051-af7c-d6d4cfdebf8e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:20.974553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:20.987463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:20.988155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:20.988442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:20.988722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:20.990193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] RBAC: Adding query filter params (interface=public, service_id=97e6f63a86c649a2b744bac33b240371, region_id=RegionOne) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:20.991159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:20.991913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:20.997572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 501/1038] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/endpoints?service_id=97e6f63a86c649a2b744bac33b240371&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:21.003042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:21.029123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afedf6-bfa3-4c7a-aeca-eec3ccfdab8d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.067774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.068750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.069051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.069358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.094683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:21.118557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d51690-be07-405e-bca4-f90be75cf5e2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.142822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.144295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.155295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.156197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.156651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.157131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.158031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.159719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.177457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 502/1039] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:08:21.265630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 538/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:21.269745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:21.325863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:21.382891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941947c6-c360-41bc-a48d-8d51251a80ed None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.420746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.421559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.421844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.422118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.433610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.435071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:21.452503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:21.453472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:21.454538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:21.454538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:21.454538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:21.454538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:21.454538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 503/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:21.461396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:21.516780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:21.563120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc21c3-1b37-4196-ac37-e4b40ecc4d4f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.599026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.599700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.600074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.600362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.601087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:21.601429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.602857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.615623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 539/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:21.620933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:21.742415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce5a03-b58c-4089-88af-da3fa72b5526 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.744104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:21.778932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.779664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.780033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.780322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.789451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.790891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:21.806859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:21.807871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:21.808847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 504/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:21.814075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:21.918287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54a9de-2d1c-4408-8502-1b6f4652a92b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:21.935731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:21.956641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:21.957316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:21.957689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:21.958171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:21.959384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:21.960091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:21.962051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2f6c1-c3da-406e-907a-32d4d4452350 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:21.979492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 540/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:21.981123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:22.086469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60153-8cbc-4162-9d2f-fe534daae848 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:22.121140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:22.121834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:22.122119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:22.122396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:22.123220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] RBAC: Adding query filter params (role.id=f206887ec47b42c0901d2d75ea1cb23d, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:22.123535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:22.124978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6cea1-95bb-4b7f-bb3d-0ab1d1358142 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:22.137711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 505/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:21 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=f206887ec47b42c0901d2d75ea1cb23d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:22.487993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493cad-30f5-4938-a95f-19b7eb57ebb6 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:22.488399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493cad-30f5-4938-a95f-19b7eb57ebb6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:22.488617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa493cad-30f5-4938-a95f-19b7eb57ebb6 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:22.490714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 541/1046] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:22.496004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:22.496300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:22.496586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:22.514094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:22.515276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:22.537396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:22.585400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 506/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:22.609290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa493cad-30f5-4938-a95f-19b7eb57ebb6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:22.736277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa493cad-30f5-4938-a95f-19b7eb57ebb6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:22.776681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:22.777437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:22.777724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:22.777995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:22.789519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:22.797183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:22.799294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:22.800353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:22.803579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 542/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:22.804843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:22.937523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6243f13-85d1-4d9f-b6f2-186217343406 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:22.974812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:22.975423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:22.975423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:22.975738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:22.976366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:22.976674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:22.978163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:22.993601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 507/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:22.995262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.033921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.119750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2a1f6-74e0-429a-aa7c-344bb1ce983b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.172741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.173978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.174401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.174785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] PATH_INFO: `/v3/users/872475215f744a2aad94f918e472edb0` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.191663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] RBAC: Authorizing `identity:get_user(user_id=872475215f744a2aad94f918e472edb0)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.193865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334a7b4-8bbb-4d8d-a244-d2f644e15cab None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.195836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 543/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:22 2022] GET /identity/v3/users/872475215f744a2aad94f918e472edb0 => generated 285 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:23.201448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.225783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.327621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb9cf5-88e6-4579-ae76-d20e10d0d137 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.363712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.364474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.364764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.365045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.434242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.435713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.453746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e97e-bd09-40e7-b46c-e23b2e9025a9 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.454029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e97e-bd09-40e7-b46c-e23b2e9025a9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.454316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616e97e-bd09-40e7-b46c-e23b2e9025a9 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.454506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.455956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 544/1051] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:23.461631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.461926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.462209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.480046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:23.481466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:23.543124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 508/1052] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:08:23.548777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.553530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 545/1053] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:23.577904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.602423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.666578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9f947-0833-46af-a77d-ee60ec6485a3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.684043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8352a-e624-4d59-a0e6-ad3e83b609c8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.685839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.702584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.703434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.703834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.704111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.705292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.706765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.709249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 509/1054] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:23.715147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.726591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.727384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.727667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.727942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.728591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.729994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:23.744281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:23.745578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 16:08:23.746446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:23.750655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 546/1055] 10.222.0.42 () {56 vars in 1105 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:23.751500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:23.827569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1a647-c44a-4cba-8d8a-608a9579d3fc None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.847305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:23.863257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9834c83-8bdb-4a3f-b24a-3646be8b9ca2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:23.866316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.866979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.867267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.867548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.868406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=35505114eb414ebc82a796d52221c265, user.id=872475215f744a2aad94f918e472edb0) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:23.868744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.870322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.876181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 510/1056] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/role_assignments?user.id=872475215f744a2aad94f918e472edb0&scope.project.id=35505114eb414ebc82a796d52221c265&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:23.923292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:23.923973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:23.924255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:23.924536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:23.925260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] RBAC: Adding query filter params (name=image) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:23.925621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:23.927347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:23.940518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 547/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:23.943129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:24.002363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:24.061820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c04f8a-b7da-46ad-81c7-2a058e368586 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:24.097677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:24.098295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.098575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.098853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.099562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] RBAC: Adding query filter params (type=image) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:24.099868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:24.101246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b7ab-0d8d-49e9-8603-b17e2ea3e062 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:24.106410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 511/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:24.112209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:24.222305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243433-ded7-4589-8a45-478807f22f50 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:24.257068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:24.257832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.258118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.258399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.259142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:24.260525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b65983-0c5d-4b56-8003-cbf3038a8d96 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:24.331102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 548/1059] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:08:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:24.362788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfc425-82cb-4018-8833-26cb9ec1818a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.363356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfc425-82cb-4018-8833-26cb9ec1818a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.363610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfc425-82cb-4018-8833-26cb9ec1818a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.365513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 512/1060] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:24.371332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.371676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.371910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.384593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:24.390726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:24.391990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:24.483191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 549/1061] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:24.506144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfc425-82cb-4018-8833-26cb9ec1818a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:24.620400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfc425-82cb-4018-8833-26cb9ec1818a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:24.635877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:24.656956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:24.657818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.658102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.658688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.671117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:24.679687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:24.680662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:24.681943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 513/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:08:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:24.687145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:24.797119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46345d3b-1f5f-4dfe-90e8-265b0b542949 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:24.828283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:24.834222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:24.835052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:24.835234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:24.835515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:24.836331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:24.836732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:24.838142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:24.847477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 550/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:08:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:24.852627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:24.999741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd0c4b-e32e-41f1-998b-bb8d982f485b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.037215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.038071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.038359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.038641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.049594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.051082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:25.071997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:25.073090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:25.074371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:25.074371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:25.074371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:25.074371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:25.074371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 514/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:25.080369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:25.188233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350cbc-ae6a-4338-b76e-01288fdd585c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.222242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.222905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.223295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.224053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.224774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:25.225086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.226490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.238339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 551/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:25.243957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:25.272273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:25.347366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a81a7-0fe1-4163-b479-a5893b78b6f0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.382202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.383079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.383373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.383651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.391337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.392831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:25.406664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:25.407645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:25.408846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:25.413713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 515/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:25.415232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:25.473732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:25.519962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e586-0bb2-4aeb-a1a3-f2dde2b20f7d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.556738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.557435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.557722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.557995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.558836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:25.559147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.560575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.583675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 552/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:25.585331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:25.675978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:25.692590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9850d-1ceb-43b9-b89e-8d23714914f7 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.727181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.727901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.728187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.728465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.739557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.740810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:25.758555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:25.759482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:25.760365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:25.760365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:25.760365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:25.760365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:25.761662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 516/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:25.765576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:25.876537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e183caf-462f-4428-b5b5-1f975c9afdbf None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:25.889939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:25.912321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:25.912966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:25.913252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:25.913557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:25.914288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:25.914602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:25.916019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422925d-5a21-4cab-a425-a572a045aa7a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:25.931669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 553/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:25.936813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.049578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e87366-7d7e-45f1-92c9-6b5e490b0856 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.085878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.086678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.086964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.087242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.094430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.095954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.097622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:26.111252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:26.116516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:26.117863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 517/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:26.118494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56c59-4c39-4cb1-9463-35569ecf58e1 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.118494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56c59-4c39-4cb1-9463-35569ecf58e1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.118494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56c59-4c39-4cb1-9463-35569ecf58e1 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.119712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 554/1071] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:26.119948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.124902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.125192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.125503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.142768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:26.143964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:26.227117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 555/1072] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:26.228465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87ddc7-1373-4ddf-a861-6eaab410a55f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.249727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.266113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.266787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.267076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.267358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.268146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:26.268516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.270074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.288340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 518/1073] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:26.289446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.360031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:26.362199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6ef34-8c7f-4f01-a78b-90def26143e2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.364525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:26.398301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.399104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.399398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.399684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.405568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd98056f-32e9-40dd-a603-832caf84a2ee None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.409935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:26.418304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:26.421276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:26.422381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 556/1074] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:26.425499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.446338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.447488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.447773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.448048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/f206887ec47b42c0901d2d75ea1cb23d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.450539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=cb545deab49645c5947c3eacd618dc07, role_id=f206887ec47b42c0901d2d75ea1cb23d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.451972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.530230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 519/1075] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:08:26 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/cb545deab49645c5947c3eacd618dc07/roles/f206887ec47b42c0901d2d75ea1cb23d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:08:26.546294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdbb94-5a07-4dd5-97ae-914c47ebc5c8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.586558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.587248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.587533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.587865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.588645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:26.588961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.590428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f047780-ab96-440d-b5fa-9ca458f736e6 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.606019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 557/1076] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:26.607414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.680290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f794b-a4cd-4e94-aab6-4b4ee7f6f373 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.680579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f794b-a4cd-4e94-aab6-4b4ee7f6f373 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.680777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f794b-a4cd-4e94-aab6-4b4ee7f6f373 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.682474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 558/1077] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:08:26.687832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.688133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.688419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.708447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:26.709804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:26.724304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e348d-5982-4d8a-bb0f-248735269543 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.736594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:26.743422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:26.761259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.762218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.762508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.762788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] PATH_INFO: `/v3/users/872475215f744a2aad94f918e472edb0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.764052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] RBAC: Authorizing `identity:get_user(user_id=872475215f744a2aad94f918e472edb0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.765536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.767321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 520/1078] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/users/872475215f744a2aad94f918e472edb0 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:26.774089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.797258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 559/1079] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:26.820755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.896061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4a00b-dc19-4bd9-bec5-1cf3e993aa23 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.935040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.935912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.936205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.936678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:26.937703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:26.939163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:26.941020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 521/1080] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:26.945644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d9569-05dc-484a-a12a-2e90363f7b4b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:26.947381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:26.983363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:26.984323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:26.984792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:26.985298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:27.008434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:27.012519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:27.058789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d459-4e67-4c43-8eca-3897b4879e0b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:27.062373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:27.063849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:27.100355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:27.101418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:27.101686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:27.101955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265/users/872475215f744a2aad94f918e472edb0/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:27.103902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] RBAC: Authorizing `identity:create_grant(project_id=35505114eb414ebc82a796d52221c265, user_id=872475215f744a2aad94f918e472edb0, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:27.105399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:27.166064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 560/1081] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:08:27.171824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:27.197757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 522/1082] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:08:26 2022] PUT /identity/v3/projects/35505114eb414ebc82a796d52221c265/users/872475215f744a2aad94f918e472edb0/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:08:27.234479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:27.287903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe25f4-6de3-42d2-a6b2-763b219f9a94 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:27.324996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:27.325845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:27.326131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:27.326409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:27.339289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:27.340762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-0f989d70-1b67-4c51-a3ab-7bee73eccfa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:27.359239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:27.360222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:27.361113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:27.361113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:27.361113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:27.361113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:27.361113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 561/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:27.367217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:27.473695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368f56-f497-45ec-a008-6b5b9427a66b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:27.487033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:27.508974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:27.509672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:27.509955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:27.510288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:27.511279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:27.511593 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:27.513018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b41ab-41f8-4ab7-b1c0-e2bac6d971e8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:27.533421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 523/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:28.786614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7bbd-c612-4715-a12c-b012aa473c69 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:28.787326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7bbd-c612-4715-a12c-b012aa473c69 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:28.787326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d7bbd-c612-4715-a12c-b012aa473c69 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:28.788609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 562/1085] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:28.793820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:28.794222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:28.794493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:28.812095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:28.813278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:28.879631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 524/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:28.903564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d7bbd-c612-4715-a12c-b012aa473c69 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.009681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d7bbd-c612-4715-a12c-b012aa473c69 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.016648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.045032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.045800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.046089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.046368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.058329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:29.065825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:29.067209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 16:08:29.068615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:29.072009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 563/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Fri May 27 16:08:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:29.073230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.173622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5926a7-bfe4-4f93-a75d-d3d49736b246 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.175870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.206527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.207194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.208011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.208330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.209088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:29.209712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.211379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.219984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 525/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:29.227391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.331728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546f48-1638-408d-8ab1-b27928a5789d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.366460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.367260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.367548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.367829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.378660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.381310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-89634de1-8209-4d98-beb2-797b0ec02b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:29.400909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:29.402103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:29.403038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:29.403038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:29.403038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:29.403038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:29.403038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 564/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:29.408287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.429694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dab6f-b5aa-4127-b8ea-3a2e2fb7269d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.430167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dab6f-b5aa-4127-b8ea-3a2e2fb7269d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.430657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950dab6f-b5aa-4127-b8ea-3a2e2fb7269d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.432500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 565/1090] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:29.439135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.439432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.439716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.459455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:29.460690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:29.475004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.525231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc790e-e779-417a-8b1a-8ce39684c1c8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.537056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 566/1091] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:29.561412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.562985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.563649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.563936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.564212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.564941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:29.565258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.567722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.581262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 526/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:29.588008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.619167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.675300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bc04-9bbc-442a-8ec6-6878a9678d9c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.703749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d79d5f-62bd-4069-a01e-32a136a1dca0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.712760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.713583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.713876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.714154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.724756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:29.732972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:29.733962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:29.734808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 567/1093] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:29.741294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.744353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.745151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.745464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.745862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.753125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.753309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.755817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:08:29.771344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:29.772377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:29.773312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 527/1094] 10.222.0.42 () {56 vars in 1093 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:29.780368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.860834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae40c8-b3b4-4a93-b284-bb2293343d8c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.875978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:29.891858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.892535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.892817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.893098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.893910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:29.894227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.895662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.895783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661476c-9837-4432-8fa5-d9e05f0b8262 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:29.905308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 568/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:29.911505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:29.939395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:29.940077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:29.940364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:29.940645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:29.941549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:29.942625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:29.944418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:29.965241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 528/1096] 10.222.0.42 () {56 vars in 1133 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:29.970919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.001427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.033822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08236dd-42ce-414e-a5e5-59caea51741c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.076330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.077404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.077947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.078356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] PATH_INFO: `/v3/users/872475215f744a2aad94f918e472edb0` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.079611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] RBAC: Authorizing `identity:get_user(user_id=872475215f744a2aad94f918e472edb0)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.081063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.082743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 569/1097] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/users/872475215f744a2aad94f918e472edb0 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:30.087836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e04fc8-eb94-4106-a33b-0dd129af744e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.088802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.123793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.124404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.124689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.124966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.140976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.195288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3676e-6568-4a8a-bbb5-421b79b24d84 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.201327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.202824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.229636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.230287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.230571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.230844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.240426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.300174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.303096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.305847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 529/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:08:30.311808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.361435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.418797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66bd7-50b5-436f-aabc-57ef6297d753 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.430981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 570/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:08:30.437167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.454443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.455242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.455526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.455812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.466178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.467672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.479647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:08:30.491714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:30.492920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:30.493895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:08:30.493895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:08:30.493895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:08:30.493895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:30.493895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 530/1100] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:30.499165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.544806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633817ae-cf3d-4345-b999-271770b6259b None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.588073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.588936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.589227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.589539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.590721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.592205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.593943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 571/1101] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:30.600000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.619345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.625797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c40f6-dd79-4904-be0e-ef3713810f6f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.661928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.662595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.662886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.663170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.663894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:30.664214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.665907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.677976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 531/1102] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:30.683439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:30.706353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef3ae0-d21a-43f9-8bfa-62cb55922d76 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.714728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:30.742429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.743084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.743367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.743644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.744468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=35505114eb414ebc82a796d52221c265, user.id=872475215f744a2aad94f918e472edb0) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:30.744793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.746300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.752341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 572/1103] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/role_assignments?user.id=872475215f744a2aad94f918e472edb0&scope.project.id=35505114eb414ebc82a796d52221c265&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:30.821764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3681a6c-29d3-4190-9833-285534cc0096 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:30.859842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:30.860652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:30.860939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:30.861220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:30.869848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:30.871354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:08:30.887285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:30.888737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:08:30.889736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:30.893539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 532/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:08:30.895038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:31.003746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b5144-f5d1-407c-a322-09882f1f0fcb None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:31.041580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:31.042245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:31.042534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:31.042815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:31.043607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:31.044038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:31.045518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ab8f6-a380-45a6-8bc0-49bd961efbff None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:31.058485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 573/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:31.064532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:31.088612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:31.178135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12225-7f0a-4da3-9a6d-ce4861adbfaa None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:31.212658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:31.213236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:31.213567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:31.213847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:31.214772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] RBAC: Adding query filter params (role.id=f206887ec47b42c0901d2d75ea1cb23d, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=cb545deab49645c5947c3eacd618dc07) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:31.215147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:31.217584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a608c-0886-4f7b-9495-f19cddb7d76c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:31.226578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 533/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:31 2022] GET /identity/v3/role_assignments?user.id=cb545deab49645c5947c3eacd618dc07&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=f206887ec47b42c0901d2d75ea1cb23d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:33.056083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0dda-bd7c-4b3a-9559-1ac831dc34de None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:33.056605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0dda-bd7c-4b3a-9559-1ac831dc34de None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:33.056695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf0dda-bd7c-4b3a-9559-1ac831dc34de None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:33.058047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 574/1107] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:33.063122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:33.063520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:33.063684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:33.080193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:33.081426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:33.148594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 534/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:08:33.174929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf0dda-bd7c-4b3a-9559-1ac831dc34de None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:33.206735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:33.280973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf0dda-bd7c-4b3a-9559-1ac831dc34de None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:33.319253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:33.320043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:33.320326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:33.320605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:33.321807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:33.323259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f7e71-f4a6-4a69-908e-4ed94e8a7196 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:33.324881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 575/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:33.331130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:33.343308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:33.435441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e365d0b-648c-4def-b8bb-32449124fd13 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:33.469072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:33.469690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:33.469975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:33.470254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:33.470966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:33.472423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06879e-62e1-45c8-97a4-f0004f0ac24d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:33.569446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 535/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:08:33 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:08:35.834741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585083-f6cf-4e7a-852a-ea36ccd05036 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:35.835271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585083-f6cf-4e7a-852a-ea36ccd05036 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:35.835363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce585083-f6cf-4e7a-852a-ea36ccd05036 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:35.836654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 576/1111] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:35.842048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:35.848960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:35.848960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:35.859640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:35.860813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:35.917915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:35.924925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 536/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:35.947936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585083-f6cf-4e7a-852a-ea36ccd05036 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:36.044433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:36.048939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce585083-f6cf-4e7a-852a-ea36ccd05036 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:36.082383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:36.083299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:36.083661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:36.083945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] PATH_INFO: `/v3/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:36.085927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] RBAC: Authorizing `identity:get_role(role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:36.087297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:36.088647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 577/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:35 2022] GET /identity/v3/roles/4d146a1b44df454bbbc17abada36747a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:36.094468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:36.190157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3687-226b-4cb9-aa28-8ed6f8c80a2a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:36.224037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:36.224887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:36.225174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:36.225490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] PATH_INFO: `/v3/users/04307de925f6464ea5d695ca0951f598` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:36.238815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] RBAC: Authorizing `identity:get_user(user_id=04307de925f6464ea5d695ca0951f598)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:36.240298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:36.241957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 537/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:36 2022] GET /identity/v3/users/04307de925f6464ea5d695ca0951f598 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:36.247684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:36.349011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bb88-deed-4328-a88e-bafa0102a761 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:36.382676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:36.383321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:36.383612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:36.383895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:36.416548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:36.450784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:36.452259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:36.548725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 578/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:36.554788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:36.608942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:36.651930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21085e57-15e3-4f7a-92d1-c4a5ffe42062 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:36.685306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:36.686179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:36.686468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:36.686748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:36.687916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:36.689373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d0330-a0ed-4d7f-8ac8-5e25ce33374f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:36.691077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 538/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:36 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:36.697299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:36.760687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:36.795951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2056a-d136-4c7a-ab04-bd6b6fbe0399 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:36.829002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:36.829685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:36.829968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:36.830245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:36.831063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=35505114eb414ebc82a796d52221c265, user.id=04307de925f6464ea5d695ca0951f598) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:36.831376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:36.832959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576bbcf-3fc9-4ac3-a3d3-4a351299538e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:36.838236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 579/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:36 2022] GET /identity/v3/role_assignments?user.id=04307de925f6464ea5d695ca0951f598&scope.project.id=35505114eb414ebc82a796d52221c265&role.id=4d146a1b44df454bbbc17abada36747a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:39.007828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbee9a-38bd-4772-a631-842c05f55551 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.008378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbee9a-38bd-4772-a631-842c05f55551 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.008378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbee9a-38bd-4772-a631-842c05f55551 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.009859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 539/1118] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:39.015319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.015719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.015878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.033061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:39.034275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:39.098882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 580/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:39.121090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbee9a-38bd-4772-a631-842c05f55551 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:39.222179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbee9a-38bd-4772-a631-842c05f55551 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:39.244479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:39.256296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:39.258053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.258053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.258053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] PATH_INFO: `/v3/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.259232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] RBAC: Authorizing `identity:get_role(role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:39.260607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:39.262023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 540/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:39 2022] GET /identity/v3/roles/4d146a1b44df454bbbc17abada36747a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:39.267804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:39.327095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:39.367199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0482d-8f82-4687-a9b4-bb2564167494 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:39.400523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:39.401467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.401701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.401981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] PATH_INFO: `/v3/users/04307de925f6464ea5d695ca0951f598` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.403166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] RBAC: Authorizing `identity:get_user(user_id=04307de925f6464ea5d695ca0951f598)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:39.404603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:39.406257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 581/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:39 2022] GET /identity/v3/users/04307de925f6464ea5d695ca0951f598 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:39.412373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:39.510226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a6f16-6ffc-47d1-85e0-3c88400c1bf6 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:39.543719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:39.544559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.544842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.545117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.546307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:39.547737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de73b1-54ef-4441-8bb4-5a3c6ae3e073 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:39.549427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 541/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:39 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:39.555988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:39.653614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5653c-5e78-4d0b-bb91-6b492b096b74 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:39.679441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:39.686922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:39.692593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:39.692593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:39.692593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265/users/04307de925f6464ea5d695ca0951f598/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:39.692593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=35505114eb414ebc82a796d52221c265, user_id=04307de925f6464ea5d695ca0951f598, role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:39.692593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40dd4d-61ec-485c-991a-b50d8cc8d4e0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:39.763047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 582/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:08:39 2022] PUT /identity/v3/projects/35505114eb414ebc82a796d52221c265/users/04307de925f6464ea5d695ca0951f598/roles/4d146a1b44df454bbbc17abada36747a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:08:41.918023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d6b7-a14a-4d12-89eb-1d6e96c36555 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:41.918571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d6b7-a14a-4d12-89eb-1d6e96c36555 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:41.918571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d6b7-a14a-4d12-89eb-1d6e96c36555 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:41.919980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 542/1124] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:41.925868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:41.926469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:41.926469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:41.943524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:41.944713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:42.010255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 583/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:42.033901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9d6b7-a14a-4d12-89eb-1d6e96c36555 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:42.093246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:42.135548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9d6b7-a14a-4d12-89eb-1d6e96c36555 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:42.170094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:42.170886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:42.171174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:42.171454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] PATH_INFO: `/v3/roles/4d146a1b44df454bbbc17abada36747a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:42.172920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] RBAC: Authorizing `identity:get_role(role_id=4d146a1b44df454bbbc17abada36747a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:42.174324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:42.176520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 543/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:42 2022] GET /identity/v3/roles/4d146a1b44df454bbbc17abada36747a => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:42.182293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:42.277461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfc87d-6677-4320-a084-fb9bad3b83fa None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:42.295479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:42.312265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:42.313127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:42.313435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:42.313710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] PATH_INFO: `/v3/users/04307de925f6464ea5d695ca0951f598` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:42.314881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] RBAC: Authorizing `identity:get_user(user_id=04307de925f6464ea5d695ca0951f598)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:42.316596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:42.318223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 584/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:42 2022] GET /identity/v3/users/04307de925f6464ea5d695ca0951f598 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:42.323961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:42.421028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b14db5-782c-4d09-aa36-cd19d71ca54e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:42.446938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:42.455170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:42.455805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:42.458444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:42.458444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:42.521865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:42.523307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:42.618898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 544/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:42.624784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:42.726213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54d939-b925-485b-85b8-2cf330ca991d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:42.759715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:42.760571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:42.760858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:42.761136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] PATH_INFO: `/v3/projects/35505114eb414ebc82a796d52221c265` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:42.762643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] RBAC: Authorizing `identity:get_project(project_id=35505114eb414ebc82a796d52221c265)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:42.764077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6f16e-759f-4321-bb66-fa1dd60a0a45 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:42.765784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 585/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:42 2022] GET /identity/v3/projects/35505114eb414ebc82a796d52221c265 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:42.772606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:42.798576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:42.868375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e7bf-584f-4b7f-8770-85fe5ed7233f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:42.901755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:42.902397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:42.902679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:42.902954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:42.903770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] RBAC: Adding query filter params (role.id=4d146a1b44df454bbbc17abada36747a, scope.project.id=35505114eb414ebc82a796d52221c265, user.id=04307de925f6464ea5d695ca0951f598) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:42.904089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:42.905530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739fa56-c039-4f7b-8750-22adf4f0c396 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:42.911543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 545/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:42 2022] GET /identity/v3/role_assignments?user.id=04307de925f6464ea5d695ca0951f598&scope.project.id=35505114eb414ebc82a796d52221c265&role.id=4d146a1b44df454bbbc17abada36747a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:45.134066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2c75-c265-42ea-b3b5-7629143398f9 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:45.134625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2c75-c265-42ea-b3b5-7629143398f9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:45.134625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2c75-c265-42ea-b3b5-7629143398f9 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:45.135963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 586/1131] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:45.141102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:45.141540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:45.141698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:45.158896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:45.160200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:45.226071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 546/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:45.248877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2c75-c265-42ea-b3b5-7629143398f9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:45.305369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:45.349289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2c75-c265-42ea-b3b5-7629143398f9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:45.381845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:45.382623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:45.382907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:45.383184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:45.384339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:45.385792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17129b-c825-453f-94df-1b5d5c720bc8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:45.387359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 587/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:45.393211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:45.416114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:45.494533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c24c4-1751-4d9e-8652-12db97f1f621 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:45.528629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:45.529279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:45.529610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:45.529891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:45.530590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:45.532288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fd99d-6f8e-4af7-a432-7ef1f1507707 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:45.604680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 547/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:08:45 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:47.852420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3710-aa44-4ea6-8a1b-75274fcd4cb1 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:47.852976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3710-aa44-4ea6-8a1b-75274fcd4cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:47.852976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3710-aa44-4ea6-8a1b-75274fcd4cb1 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:47.854367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 588/1135] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:47.859652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:47.860013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:47.860217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:47.877464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:47.879476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:47.944504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 548/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:47.969466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c3710-aa44-4ea6-8a1b-75274fcd4cb1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:48.068854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c3710-aa44-4ea6-8a1b-75274fcd4cb1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:48.091181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:48.102180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa114dc8-8250-42bf-8370-4b4bbf29dce0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:48.109988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 589/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:08:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:48.113805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:48.115979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:48.212176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5c9f-cdfe-4343-b1c9-2121133efe2e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:48.252601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:48.254105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:48.254260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:48.254415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:48.255117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:48.256557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc0555-c0e5-4d3a-97e0-2d2fe2a6bf2a None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:48.381065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 549/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:08:48 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:08:50.621038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23e498-2f27-4934-9e1e-27853271427d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:50.621722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23e498-2f27-4934-9e1e-27853271427d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:50.621722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23e498-2f27-4934-9e1e-27853271427d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:50.622965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 590/1139] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:50.628240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:50.628657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:50.628810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:50.645720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:50.646899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:50.698585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:50.712316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 550/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:50.737170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23e498-2f27-4934-9e1e-27853271427d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:50.837854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23e498-2f27-4934-9e1e-27853271427d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:50.872965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:50.873702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:50.873991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:50.874272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:50.884315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:50.891859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:50.893000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:50.894816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 591/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:50.899757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:50.999916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7fee56-f8cb-4d42-a0d7-5c6bc5c3d7f4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:51.032682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:51.033363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:51.033650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:51.034746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:51.035265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:51.035575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:51.036943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:51.044731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 551/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:51.051284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:51.099633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:51.147649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f773d-d4eb-4ed9-adcc-60ae2c03e852 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:51.180504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:51.181370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:51.181661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:51.181935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] PATH_INFO: `/v3/users/1eaf9a10b12745a2893300de4d5639a7` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:51.195740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] RBAC: Authorizing `identity:get_user(user_id=1eaf9a10b12745a2893300de4d5639a7)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:51.197185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:51.198820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 592/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:51 2022] GET /identity/v3/users/1eaf9a10b12745a2893300de4d5639a7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:51.204498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:51.254833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:51.304818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e6d73-6248-4d22-b768-85e9c1c6c363 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:51.339031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:51.339674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:51.339962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:51.340243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:51.408046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:51.409920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:51.456713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:51.505577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 552/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 16:08:51.511495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:51.606404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c711b-c8da-44ff-af19-f988045c0a0c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:51.616122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:51.639595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:51.640444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:51.640730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:51.641008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] PATH_INFO: `/v3/projects/39841a9ebc654cdaa22ba20ac379c360` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:51.642202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] RBAC: Authorizing `identity:get_project(project_id=39841a9ebc654cdaa22ba20ac379c360)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:51.644309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20280ac3-662c-41af-8e4f-13960c4af22e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:51.646029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 593/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:51 2022] GET /identity/v3/projects/39841a9ebc654cdaa22ba20ac379c360 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:51.652191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:51.748551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0796466-0880-4cfe-acbf-455d26815112 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:51.782463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:51.783093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:51.783371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:51.783643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:51.784609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=39841a9ebc654cdaa22ba20ac379c360, user.id=1eaf9a10b12745a2893300de4d5639a7) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:51.784920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:51.786377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f80242-ab69-46d2-968d-f30542846301 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:51.791754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 553/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:51 2022] GET /identity/v3/role_assignments?user.id=1eaf9a10b12745a2893300de4d5639a7&scope.project.id=39841a9ebc654cdaa22ba20ac379c360&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:53.963783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c397-4356-4af7-ba25-a188bf238f27 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:53.964330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c397-4356-4af7-ba25-a188bf238f27 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:53.964330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c397-4356-4af7-ba25-a188bf238f27 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:53.965730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 594/1147] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:53.970843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:53.971832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:53.972041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:53.988929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:53.990165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:53.999215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:54.057405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 554/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:54.081617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c397-4356-4af7-ba25-a188bf238f27 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:54.180552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c397-4356-4af7-ba25-a188bf238f27 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:54.214011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:54.214726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:54.215010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:54.215286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:54.225037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:54.232519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:54.233717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:08:54.234727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 595/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:54.239821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:54.341402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620286c-1d0b-4d22-b9d2-a6051313e4cb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:54.374287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:54.374945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:54.375231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:54.375510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:54.376243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:54.376552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:54.378628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:54.386516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 555/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:54.392538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:54.428985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:54.490748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8137-4ce8-41d2-b098-75bf41435571 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:54.523577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:54.524437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:54.524723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:54.525001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] PATH_INFO: `/v3/users/1eaf9a10b12745a2893300de4d5639a7` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:54.526205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] RBAC: Authorizing `identity:get_user(user_id=1eaf9a10b12745a2893300de4d5639a7)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:54.527639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:54.529232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 596/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:54 2022] GET /identity/v3/users/1eaf9a10b12745a2893300de4d5639a7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:54.535507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:54.544515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:54.634938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23079579-8650-40fb-821d-aa70b84ba785 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:54.667180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:54.668022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:54.668299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:54.668570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] PATH_INFO: `/v3/projects/39841a9ebc654cdaa22ba20ac379c360` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:54.669744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] RBAC: Authorizing `identity:get_project(project_id=39841a9ebc654cdaa22ba20ac379c360)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:54.671169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edc084-2908-4b1b-a161-e9a7ef2d7297 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:54.672836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 556/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:54 2022] GET /identity/v3/projects/39841a9ebc654cdaa22ba20ac379c360 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:08:54.679464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:54.778032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a887be3-c799-4ab3-97f9-d79b0664f89c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:54.780245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:54.812245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:54.813128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:54.813444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:54.813732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] PATH_INFO: `/v3/projects/39841a9ebc654cdaa22ba20ac379c360/users/1eaf9a10b12745a2893300de4d5639a7/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:54.815813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] RBAC: Authorizing `identity:create_grant(project_id=39841a9ebc654cdaa22ba20ac379c360, user_id=1eaf9a10b12745a2893300de4d5639a7, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:54.817248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae090-2c3c-449d-be7b-baf4a5c8bafc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:54.882293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 597/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:08:54 2022] PUT /identity/v3/projects/39841a9ebc654cdaa22ba20ac379c360/users/1eaf9a10b12745a2893300de4d5639a7/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:08:57.064143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352a97-1ea4-4fbc-8b8c-736322dd5f23 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.064689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352a97-1ea4-4fbc-8b8c-736322dd5f23 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.064791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352a97-1ea4-4fbc-8b8c-736322dd5f23 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.066190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 557/1154] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:08:57.071346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.071723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.071924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.088310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:08:57.089530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:08:57.154067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 598/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:08:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:08:57.178696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352a97-1ea4-4fbc-8b8c-736322dd5f23 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:57.254392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:57.279014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352a97-1ea4-4fbc-8b8c-736322dd5f23 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:57.313965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:57.314776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.315056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.315330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.325263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:08:57.333304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:08:57.334371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:08:57.335239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 558/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:08:57.340591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:57.431126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:57.437503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34ebd-8c1c-41bb-8751-bbcdf16bb1c9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:57.471753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:57.472504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.472840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.473063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.473928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:57.474211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:57.475589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:57.484125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 599/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:57.489670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:57.589510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2ea2b-89e6-4b97-ba09-3cc3d289c349 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:57.622634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:57.623492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.623775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.624054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] PATH_INFO: `/v3/users/1eaf9a10b12745a2893300de4d5639a7` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.625226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=1eaf9a10b12745a2893300de4d5639a7)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:57.626726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:57.628638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 559/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:08:57 2022] GET /identity/v3/users/1eaf9a10b12745a2893300de4d5639a7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:57.634209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:57.729925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd54d-6970-452e-949b-c216f6bfe959 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:57.762095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:57.762726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:57.763010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:57.763291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:57.787268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:57.829728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:57.831324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:57.928049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 600/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:08:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:08:57.934112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:57.936995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:58.037364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dac48-72f5-40b1-abd0-9406dea1f0d0 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:58.069976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:58.070832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:58.071114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:58.071387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] PATH_INFO: `/v3/projects/39841a9ebc654cdaa22ba20ac379c360` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:58.072537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] RBAC: Authorizing `identity:get_project(project_id=39841a9ebc654cdaa22ba20ac379c360)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:58.074001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0eadc-e847-4d7d-a848-0df77fa4544b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:58.075669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 560/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:08:57 2022] GET /identity/v3/projects/39841a9ebc654cdaa22ba20ac379c360 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:08:58.082545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:08:58.136824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:08:58.182844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd98c3-df3f-46ee-9ab6-7eeed0eb495f None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:08:58.216379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:08:58.217024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:08:58.217308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:08:58.217623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:08:58.218445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=39841a9ebc654cdaa22ba20ac379c360, user.id=1eaf9a10b12745a2893300de4d5639a7) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:08:58.218758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:08:58.220189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a46c5-51bd-4eab-8883-a481d294c1eb None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:08:58.226636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 601/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:08:58 2022] GET /identity/v3/role_assignments?user.id=1eaf9a10b12745a2893300de4d5639a7&scope.project.id=39841a9ebc654cdaa22ba20ac379c360&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:09:00.471342 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e16c8-ecc0-440f-8778-abcde083d455 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:00.471899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e16c8-ecc0-440f-8778-abcde083d455 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:00.471899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e16c8-ecc0-440f-8778-abcde083d455 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:00.473272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 561/1162] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:00.478945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:00.479390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:00.479509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:00.495903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:00.497082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:00.560768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 602/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:00.583668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e16c8-ecc0-440f-8778-abcde083d455 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:00.651426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:00.681861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e16c8-ecc0-440f-8778-abcde083d455 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:00.713748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:00.714386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:00.714668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:00.714944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:00.778406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:00.779775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:00.843622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:00.876860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 562/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:09:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:09:00.882920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:00.931433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:00.981787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd803c-6f12-4394-9459-260c192eb0e1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:01.015534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:01.016332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:01.016621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:01.016900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:01.023732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:01.025170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b92968e4-886b-4170-a3d0-b8004761f478 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:09:01.039878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:09:01.041158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:09:01.042299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 603/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:09:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:09:01.047127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:01.149006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d70e8-5d95-4725-b4d2-d1bdce32dc8e None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:01.182543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:01.183197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:01.183484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:01.183766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:01.184475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:01.184782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:01.186185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24f535-c05c-4dbd-8660-7a7dcdbd9e84 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:01.193148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 563/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Fri May 27 16:09:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:03.372037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c844-9fbe-4492-8a0c-a6baddc01599 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:03.372547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c844-9fbe-4492-8a0c-a6baddc01599 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:03.372652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41c844-9fbe-4492-8a0c-a6baddc01599 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:03.374118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 604/1167] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:03.380093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf2b18-7925-44ef-bf90-8d5488c8a241 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:03.380557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf2b18-7925-44ef-bf90-8d5488c8a241 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:03.380656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf2b18-7925-44ef-bf90-8d5488c8a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:03.399263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3caf2b18-7925-44ef-bf90-8d5488c8a241 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:03.400470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3caf2b18-7925-44ef-bf90-8d5488c8a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:03.428387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:03.467474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 564/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:03.491171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41c844-9fbe-4492-8a0c-a6baddc01599 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:03.591664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41c844-9fbe-4492-8a0c-a6baddc01599 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:03.602084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:03.624642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:03.625233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:03.625552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:03.625831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:03.626653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:03.628635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb0d4b-d7d3-4c80-a75a-387a3831b091 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:03.694842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 605/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Fri May 27 16:09:03 2022] POST /identity/v3/domains => generated 267 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:09:05.972585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04127cf-1e0e-40bd-bc8c-03d8472fb729 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:05.973076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04127cf-1e0e-40bd-bc8c-03d8472fb729 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:05.973177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04127cf-1e0e-40bd-bc8c-03d8472fb729 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:05.974535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 565/1170] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:05.979705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:05.980105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:05.980274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:05.996551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:05.997776 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:06.061895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 606/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:09:06.085999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04127cf-1e0e-40bd-bc8c-03d8472fb729 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:06.183429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04127cf-1e0e-40bd-bc8c-03d8472fb729 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:06.215851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:06.216698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:06.216984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:06.217261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] PATH_INFO: `/v3/domains/8747f97a43a949a2bf69b98462ad5d4a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:06.227817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=8747f97a43a949a2bf69b98462ad5d4a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:06.229256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b166e2a-8bf0-4439-8248-25c34e062bdb None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:06.230890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 566/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Fri May 27 16:09:06 2022] GET /identity/v3/domains/8747f97a43a949a2bf69b98462ad5d4a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:06.236767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:06.330624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:06.333503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f24a8f-9237-45b5-820a-260a63813641 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:06.366918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:06.367561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:06.367843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:06.368119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:06.368827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:06.370348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e4120-2365-4fdc-8bb4-d91d3d30c6f7 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:06.459902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 607/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:09:06 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:09:08.767702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e464-be47-4748-a9df-9cfd364e9341 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:08.768258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e464-be47-4748-a9df-9cfd364e9341 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:08.768363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56e464-be47-4748-a9df-9cfd364e9341 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:08.769757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 567/1174] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:08.775119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:08.775520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:08.775690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:08.792930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:08.794165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:08.860297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 608/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:08.887132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e464-be47-4748-a9df-9cfd364e9341 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:08.912328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:08.988841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56e464-be47-4748-a9df-9cfd364e9341 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:09.021070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:09.021882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:09.022168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:09.022440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] PATH_INFO: `/v3/domains/8747f97a43a949a2bf69b98462ad5d4a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:09.023610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=8747f97a43a949a2bf69b98462ad5d4a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:09.025038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6848-9c22-4aa7-a7ce-b5105947a4dd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:09.026655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 568/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Fri May 27 16:09:08 2022] GET /identity/v3/domains/8747f97a43a949a2bf69b98462ad5d4a => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:09.033222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:09.110440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:09.128344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352234a7-4863-4f93-ae43-b62e6a6109ba None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:09.161694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:09.162360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:09.162647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:09.162927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:09.163636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:09.165085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591ee30-8e53-428d-a0e1-4b1c200f5bff None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:09.282987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 609/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:09:09 2022] POST /identity/v3/users => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:09:11.499234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6630-7427-4bce-9025-c383aff47998 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:11.499766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6630-7427-4bce-9025-c383aff47998 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:11.499766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6630-7427-4bce-9025-c383aff47998 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:11.501726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 569/1178] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:11.507214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:11.507639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:11.507780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:11.525525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:11.526728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:11.592913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 610/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:11.617690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6630-7427-4bce-9025-c383aff47998 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:11.707422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:11.716554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6630-7427-4bce-9025-c383aff47998 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:11.749424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:11.750323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:11.750789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:11.752221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:11.762586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:09:11.770635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:09:11.771755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:09:11.772797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 570/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:09:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:09:11.779773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:11.846559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:11.879576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7ef6a-cbf4-40af-93b6-9b1a26842447 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:11.916107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:11.916773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:11.917065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:11.917498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:11.918570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:11.919078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:11.921137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:11.931583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 611/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:09:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:11.937155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:12.037893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d17f81-6ca9-4b9f-a88b-065094c4a88f None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:12.070447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:12.071310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:12.071595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:12.071872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] PATH_INFO: `/v3/users/8c0fb4c607f5473881762ca9580d0108` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:12.085299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] RBAC: Authorizing `identity:get_user(user_id=8c0fb4c607f5473881762ca9580d0108)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:12.086836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:12.088527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 571/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:09:11 2022] GET /identity/v3/users/8c0fb4c607f5473881762ca9580d0108 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:12.094313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:12.190719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04866a-02fc-4b14-8579-19e4b071fb9d None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:12.218085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:12.223921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:12.224563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:12.224846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:12.225122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:12.292934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:12.294345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:12.388747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 612/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:09:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:09:12.395216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:12.407532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:12.492602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5601706-43e7-4ecd-b841-935373811955 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:12.525094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:12.525986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:12.526266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:12.526536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] PATH_INFO: `/v3/projects/59fff308602a4b2cb13d8129e3df95d1` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:12.527684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] RBAC: Authorizing `identity:get_project(project_id=59fff308602a4b2cb13d8129e3df95d1)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:12.529112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835fc82-0f57-4add-8e9d-2cf3563546a1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:12.531056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 572/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:09:12 2022] GET /identity/v3/projects/59fff308602a4b2cb13d8129e3df95d1 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:12.537679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:12.567489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:12.635478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68027e-833d-4534-8149-6bb48bebdf62 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:12.667950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:12.668592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:12.668875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:12.669151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:12.670001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=59fff308602a4b2cb13d8129e3df95d1, user.id=8c0fb4c607f5473881762ca9580d0108) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:12.670313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:12.671733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578ae20-d6e6-4147-ac12-b29d3c9f5fd9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:12.677658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 613/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:09:12 2022] GET /identity/v3/role_assignments?user.id=8c0fb4c607f5473881762ca9580d0108&scope.project.id=59fff308602a4b2cb13d8129e3df95d1&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:14.823119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387545df-1718-4b27-9d3e-6465a207dbb6 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:14.823679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387545df-1718-4b27-9d3e-6465a207dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:14.823679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387545df-1718-4b27-9d3e-6465a207dbb6 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:14.825039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 573/1186] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:14.830823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:14.831229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:14.831375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:14.848991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:14.857462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:14.915540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 614/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:14.939614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387545df-1718-4b27-9d3e-6465a207dbb6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:15.025643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:15.048403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387545df-1718-4b27-9d3e-6465a207dbb6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:15.082016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:15.082731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:15.083015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:15.083292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:15.093498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:09:15.101801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:09:15.102916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:09:15.103965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 574/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:09:15.111583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:15.135341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:15.209285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce146-c952-4bf2-a7b7-0efadb20a8a6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:15.243363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:15.244026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:15.244313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:15.244593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:15.245364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:15.245677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:15.247054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:15.254814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 615/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:15.260724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:15.362355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9c9b1-014a-47dc-b418-e8c941137022 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:15.397878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:15.398755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:15.399043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:15.399322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] PATH_INFO: `/v3/users/8c0fb4c607f5473881762ca9580d0108` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:15.400516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] RBAC: Authorizing `identity:get_user(user_id=8c0fb4c607f5473881762ca9580d0108)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:15.401995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:15.403626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 575/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:09:15 2022] GET /identity/v3/users/8c0fb4c607f5473881762ca9580d0108 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:15.410252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:15.492683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:15.508251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1a9a7-aeb0-4b6a-9f6e-0a29ebb642cc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:15.541222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:15.542104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:15.542390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:15.542668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] PATH_INFO: `/v3/projects/59fff308602a4b2cb13d8129e3df95d1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:15.543834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] RBAC: Authorizing `identity:get_project(project_id=59fff308602a4b2cb13d8129e3df95d1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:15.545274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91be0c-900a-4264-8d6a-b8eb36687d84 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:15.546965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 616/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:09:15 2022] GET /identity/v3/projects/59fff308602a4b2cb13d8129e3df95d1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:15.553366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:15.562361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:15.652512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48feb-9e11-4f99-bd96-92bf9a25d291 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:15.684800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:15.685703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:15.685986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:15.686259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] PATH_INFO: `/v3/projects/59fff308602a4b2cb13d8129e3df95d1/users/8c0fb4c607f5473881762ca9580d0108/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:15.689043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] RBAC: Authorizing `identity:create_grant(project_id=59fff308602a4b2cb13d8129e3df95d1, user_id=8c0fb4c607f5473881762ca9580d0108, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:15.690920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c4d70-85bd-4617-ae69-ddf0df171e6f None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:15.772711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 576/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:09:15 2022] PUT /identity/v3/projects/59fff308602a4b2cb13d8129e3df95d1/users/8c0fb4c607f5473881762ca9580d0108/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:09:17.947345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e6d7-92f7-4ee5-8d1f-3a2760b91c78 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:17.947807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e6d7-92f7-4ee5-8d1f-3a2760b91c78 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:17.947900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e6d7-92f7-4ee5-8d1f-3a2760b91c78 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:17.949252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 617/1193] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:09:17.955476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:17.955882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:17.956039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:17.973144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:09:17.974361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:09:18.032314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:18.039575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 577/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:09:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:09:18.063640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e6d7-92f7-4ee5-8d1f-3a2760b91c78 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:18.168446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e6d7-92f7-4ee5-8d1f-3a2760b91c78 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:18.202919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:18.204221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:18.205070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:18.205377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:18.215328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:09:18.223396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:09:18.224488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:09:18.225573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 618/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:09:18.230777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:18.326758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaa5e9-e0a0-4315-80a3-be1c35fe5ede None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:18.360363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:18.361364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:18.361661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:18.361940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:18.362682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:18.362992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:18.364378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:18.372402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 578/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:18.379261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:18.395358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:18.476623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73c5d7-2d42-4cc5-b8e6-a39b8f8d1b82 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:18.510511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:18.511373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:18.511662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:18.511944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] PATH_INFO: `/v3/users/8c0fb4c607f5473881762ca9580d0108` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:18.513121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] RBAC: Authorizing `identity:get_user(user_id=8c0fb4c607f5473881762ca9580d0108)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:18.514615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:18.516267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 619/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/users/8c0fb4c607f5473881762ca9580d0108 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:18.522161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:18.580606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:18.621296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ee68d-5381-4461-b432-bf9b9aa66dbe None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:18.654306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:18.655207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:18.655488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:18.655762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:18.721683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:18.723148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:18.776974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:18.817658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 579/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:09:18.823381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:18.902859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:18.921041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd079-3b61-44a5-b6d5-2481c20442ca None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:18.955840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:18.956694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:18.956976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:18.957251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] PATH_INFO: `/v3/projects/59fff308602a4b2cb13d8129e3df95d1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:18.958445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] RBAC: Authorizing `identity:get_project(project_id=59fff308602a4b2cb13d8129e3df95d1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:18.959879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e61ed5-a9ef-4925-9f66-9e36ff0446ca None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:18.961566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 620/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/projects/59fff308602a4b2cb13d8129e3df95d1 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:18.967648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:09:19.063930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525aa0f-28c0-427b-a5af-e7707f4ad0a1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:09:19.098466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:09:19.099111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:09:19.099391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:09:19.099665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:09:19.100478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=59fff308602a4b2cb13d8129e3df95d1, user.id=8c0fb4c607f5473881762ca9580d0108) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:09:19.100790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:09:19.102238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056b1ca-ff8e-4e15-a5f8-048dbb07c541 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:09:19.108039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 580/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:09:18 2022] GET /identity/v3/role_assignments?user.id=8c0fb4c607f5473881762ca9580d0108&scope.project.id=59fff308602a4b2cb13d8129e3df95d1&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:09:28.951087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=25 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:09:29.105295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:39.033136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3ce-9cdb-447b-b004-c425b93de049 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:39.033764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3ce-9cdb-447b-b004-c425b93de049 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:39.033764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3ce-9cdb-447b-b004-c425b93de049 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:39.035097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 621/1201] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:10:39.040444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:39.040910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:39.041034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:39.064031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:10:39.065274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:10:39.146079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 581/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:10:39.174017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b3ce-9cdb-447b-b004-c425b93de049 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:39.293718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b3ce-9cdb-447b-b004-c425b93de049 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:39.330778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:39.331594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:39.331882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:39.332157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:39.345221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:39.347272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:39.367483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:39.369174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:39.370797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:39.370797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:39.370797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:39.370797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:39.370797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 622/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:39.375267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:39.492190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df72d53-c043-41e1-a428-c5c0428e819d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:39.527621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:39.534690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:39.535583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:39.535910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:39.536289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:39.537229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:39.537674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:39.539674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8323a-0704-45ff-9fc7-72b4239eccee None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:39.553286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 582/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:39.559032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:39.622029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:39.673616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39c961-f9fc-4d06-b08b-2a77d056b1c4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:39.711670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:39.714057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:39.714057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:39.714057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:39.714455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:39.715848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6007bf8-1490-4c33-a18b-90eab06284ba None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:39.809880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 623/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Fri May 27 16:10:39 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:10:42.095736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc83415-50d3-4b94-bce6-dbb5c4e2aa02 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:42.096294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc83415-50d3-4b94-bce6-dbb5c4e2aa02 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:42.096294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc83415-50d3-4b94-bce6-dbb5c4e2aa02 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:42.097966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 583/1206] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:10:42.103161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:42.103606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:42.103768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:42.122498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:10:42.124949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:10:42.211295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 624/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:10:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:10:42.245769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc83415-50d3-4b94-bce6-dbb5c4e2aa02 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:42.360876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc83415-50d3-4b94-bce6-dbb5c4e2aa02 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:42.400618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:42.402265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:42.402409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:42.405702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:42.415912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:10:42.424569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:42.425977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:42.427087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 584/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:10:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:10:42.433231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:42.490124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:42.558927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39622d9-f13c-476f-b6b0-16e62effe930 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:42.593971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:42.599714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:42.606740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 625/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:10:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:42.615030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:42.685723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:42.752617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1cba5-526c-45e3-9bd7-2c6add33b815 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:42.803315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:42.804364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:42.804755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:42.805171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:42.817443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:42.818919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:42.838895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:42.843831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:42.844787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:42.844787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:42.844787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:42.844787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:42.844787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 585/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:42.846819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:42.962755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:42.966919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e68f-c726-4aa6-a415-4b6586f31f9e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:43.012294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:43.012939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:43.013250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:43.013583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:43.014415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:43.014736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:43.016142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:43.029745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 626/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:43.038583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:43.146953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf661b-c2ea-4e7a-93bd-d8a3086e38f2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:43.148510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:43.184019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:43.184835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:43.185448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:43.185923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:43.195187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:43.196684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:10:43.216558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:43.217841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:43.218915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 586/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:43.224224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:43.342820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945876d-9d06-49d0-93de-8ac488f9ebc9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:43.378258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:43.378981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:43.379265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:43.379540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:43.380383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:43.380695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:43.382168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:43.404562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 627/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:10:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:10:43.410316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:43.527696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d1758-322d-404a-97f3-562b351f36f3 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:43.570956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:43.571621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:43.571911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:43.572196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:43.606469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:43.654573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:43.656263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:43.779745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 587/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:10:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:10:43.786410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:43.809191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:43.903572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2df0c3-1b78-4a0e-954b-647a0257fd85 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:43.946764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:43.947732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:43.948018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:43.948294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:43.964697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:43.966228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:43.991262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:43.994071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:43.995801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:43.995801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:43.995801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:43.995801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:43.995801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 628/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:43.999184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:44.076782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:44.142003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ec2d-0aec-4025-bd0c-6a1c400af4d2 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:44.179216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:44.179848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:44.180077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:44.180352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:44.181081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:44.181410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:44.183580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:44.196335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 588/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:10:44.202640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:44.292773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:44.328014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42adb9-aaaf-4d5b-90d7-6eb2a776eae3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:44.378816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:44.379627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:44.379913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:44.380192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:44.388296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:44.389832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:10:44.410157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:44.411259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:44.412512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 629/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:44.418376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:44.538276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78605a-7d5e-4e77-8a5b-70d7630b934c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:44.565690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:44.579004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:44.579681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:44.579971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:44.580252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:44.581097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:44.581431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:44.582917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683a24-460a-4c2e-a46c-508639987698 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:44.597617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 589/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:44.605082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:44.724712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877a0d1-aeb4-49fa-a451-af2ca2613271 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:44.765153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:44.774489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:44.776256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:44.776745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:44.777190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:44.778596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ce1f644a909746fe92bd0dee4e9ce635) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:44.779125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:44.781034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fdee2-b555-47f6-9f23-dc5297b0f921 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:44.787082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 630/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:10:44 2022] GET /identity/v3/role_assignments?user.id=ce1f644a909746fe92bd0dee4e9ce635&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:47.053605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b0ce-913c-4726-b05c-02351e4b8adf None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.054125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b0ce-913c-4726-b05c-02351e4b8adf None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.054241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b0ce-913c-4726-b05c-02351e4b8adf None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.055533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 590/1220] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:10:47.061795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.062219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.062353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.078914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:10:47.080109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:10:47.149824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 631/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:10:47.173348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b0ce-913c-4726-b05c-02351e4b8adf None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:47.280927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b0ce-913c-4726-b05c-02351e4b8adf None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:47.314908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:47.315703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.315987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.316266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.326541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:10:47.334216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:47.335326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:47.336219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 591/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:47.342365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:47.442727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbc85f-eab6-4260-80b5-69fd30f2d9f8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:47.469295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:47.483196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:47.483885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.484219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.484511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.485281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:47.485872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:47.487013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:47.496556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 632/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:10:47.502857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:47.550562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:47.611725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8bae8-b891-4d8e-801c-4c06ee9d3aaa None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:47.648013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:47.648799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.649361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.649640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.661791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:47.663287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:47.680117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:47.681247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:47.682944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:47.682944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:47.682944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:47.682944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:47.682944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 592/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:47.687421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:47.791777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e60b2-b5c9-4b9f-8534-312b0fc5bd78 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:47.832280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:47.832280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:47.832280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:47.832280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:47.832725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:47.832725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:47.832725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:47.848219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 633/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:47.854330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:47.956665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:47.964742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b342-0dc5-4f0e-a747-48943ffaeb00 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:47.999900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.000704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.000989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.001267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.009031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.011234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:10:48.025813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:48.026779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:48.027875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 593/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:48.032971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.080508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:48.134002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7bae6-8bb2-47a5-877e-8ded02ac65a8 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:48.171451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.171451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.171451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.173073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.173073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:48.173073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.174266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.195813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 634/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:10:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:48.202409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.308340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b537e-12ed-46d8-aa29-d96bf1e29c38 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:48.343795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.344586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.344869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.345144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.358564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.360015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:48.378420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:48.379563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:48.380683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:48.380683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:48.380683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:48.380683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:48.380683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 594/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:48.385627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.487133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68c3c3-f603-49ea-9aa6-4e931d2709a0 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:48.489502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:48.521376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.522032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.522316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.522588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.523352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:48.523741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.525451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.538026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 635/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:48.545022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.554569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:48.647015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baed5c-91c5-493f-9a72-cb8754ec294d None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:48.683420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.684216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.684505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.684786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.691545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.693094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:10:48.707662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:48.708813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:48.710130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 595/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:10:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:48.715389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.823153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca6b4-6dd6-4382-a518-41205a3029f4 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:48.860643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:48.862712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:48.862712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:48.862712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:48.862712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:48.863085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:48.864482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd66e-c517-493e-bc17-472d1ca3a50c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:48.877179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 636/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:48.883617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:48.956058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:48.991640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5c0ff-4ee4-4fd8-a1cb-974176dd9ff4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:49.024827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:49.026444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:49.026728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:49.027004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ce1f644a909746fe92bd0dee4e9ce635/roles/c996e4d90a9c497f97c69a91899c9bc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:49.042624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=ce1f644a909746fe92bd0dee4e9ce635, role_id=c996e4d90a9c497f97c69a91899c9bc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:49.044116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0590-3726-4774-a110-4def2c8e73db None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:49.215385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 596/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:10:48 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ce1f644a909746fe92bd0dee4e9ce635/roles/c996e4d90a9c497f97c69a91899c9bc3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:10:51.425567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7cd44-8388-493d-94a2-779f8ac5a26d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:51.426096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7cd44-8388-493d-94a2-779f8ac5a26d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:51.426183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7cd44-8388-493d-94a2-779f8ac5a26d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:51.427569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 637/1233] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:10:51.433610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:51.434036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:51.434178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:51.450632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:10:51.451850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:10:51.521415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 597/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:10:51.551378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7cd44-8388-493d-94a2-779f8ac5a26d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:51.578533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:51.657789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7cd44-8388-493d-94a2-779f8ac5a26d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:51.691770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:51.692515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:51.692796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:51.693068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:51.702835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:10:51.710484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:51.714997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:51.716174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 638/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Fri May 27 16:10:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:51.718373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:51.766773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:51.820612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6a5f2-a988-4c8b-af2b-3040a18360cb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:51.855008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:51.855676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:51.855957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:51.856229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:51.856980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:51.857289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:51.858856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:51.867437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 598/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Fri May 27 16:10:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:51.873404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:51.959802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:51.976649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62252740-8d46-4af6-8809-0507e03f83a9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:52.012619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:52.013443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:52.013730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:52.014008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:52.024364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:52.028398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:52.043965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:52.045052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:52.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:52.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:52.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:52.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:52.046191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 639/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:52.050991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:52.152894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f5d-741b-47ad-8fdf-1929a83588ec None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:52.162349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:52.186337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:52.187001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:52.187287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:52.187562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:52.188278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:52.188587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:52.189997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:52.204018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 599/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:52.210232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:52.310505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab72d2-c879-4928-b481-9ce4041c8223 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:52.346965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:52.347789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:52.348056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:52.348333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:52.351248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:52.355958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:52.357446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:10:52.372764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:52.373969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:52.375134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 640/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:52.380060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:52.485672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd06b50-c434-4d6f-9be8-6fa8546b05fe None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:52.519731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:52.520395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:52.520678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:52.520958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:52.521815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:52.522127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:52.523829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:52.541899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 600/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:10:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:52.547941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:52.653515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab09cde-66e9-4862-881f-571f2eb2f504 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:52.689663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:52.690330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:52.690617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:52.690898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:52.754151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:52.761660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:52.763164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:52.865071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 641/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:10:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:10:52.872652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:52.904212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:52.980691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232723d2-14c3-4418-bb2c-1cac51a5eed1 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:53.018501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:53.019208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:53.019492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:53.019774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:53.030355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:53.032306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:53.051499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:53.052616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:53.053740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:53.053740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:53.053740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:53.053740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:53.053740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 601/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:53.060061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:53.158437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:53.164132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01dbcd-9be6-47d1-8f08-6ed529958cac None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:53.198333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:53.198992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:53.199273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:53.199547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:53.200263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:53.200567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:53.201974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:53.213424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 642/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:53.219161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:53.300127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:53.322926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35852501-73a0-428d-942c-1287f4fc43d4 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:53.358747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:53.359549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:53.359839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:53.360120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:53.366965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:53.369083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:10:53.383561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:53.385441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:53.386344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 602/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:53.391788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:53.492856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ae771-4eb4-4546-98dd-a64d6b32e75a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:53.527785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:53.528444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:53.528727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:53.529003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:53.529906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:53.538617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:53.538617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086745cf-381d-492b-8f05-4b9f38e135f8 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:53.543931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 643/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:10:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:53.550543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:53.652012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef76f7-c683-4f5a-a796-37541ac487ad None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:53.686602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:53.687271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:53.687552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:53.687828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:53.688791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] RBAC: Adding query filter params (role.id=c996e4d90a9c497f97c69a91899c9bc3, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ce1f644a909746fe92bd0dee4e9ce635) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:53.689013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:53.690489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e2125-5499-482c-a131-3f57481ae49b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:53.697099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 603/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:10:53 2022] GET /identity/v3/role_assignments?user.id=ce1f644a909746fe92bd0dee4e9ce635&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c996e4d90a9c497f97c69a91899c9bc3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:55.961501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f057-0851-446c-9bc9-62e9aff17832 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:55.962694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f057-0851-446c-9bc9-62e9aff17832 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:55.962694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f057-0851-446c-9bc9-62e9aff17832 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:55.964047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 644/1247] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:10:55.970033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:55.975995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:55.975995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:55.975995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:55.987912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:10:55.989100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:10:56.055677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 604/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:10:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:10:56.079882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0f057-0851-446c-9bc9-62e9aff17832 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.098628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:56.182567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0f057-0851-446c-9bc9-62e9aff17832 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:56.219829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:56.219829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:56.219829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:56.219829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:56.230102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:10:56.238947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:56.240106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:56.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 645/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:56.247039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.347058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1fb2b-6df4-4500-8608-61ef12e75928 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:56.376609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:56.381436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:56.382093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:56.382374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:56.382645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:56.383370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:56.383672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:56.385673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:56.393614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 605/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:56.399633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.479931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:56.506077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9716-0271-488f-acad-d8a6db10a0dc None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:56.539825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:56.548531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:56.548531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:56.548531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:56.552018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:56.553518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:56.570614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:56.572403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:56.573815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:56.573815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:56.573815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:56.573815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:56.573815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 646/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:56.582006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.679882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cd649-5ba4-4675-a2ff-b819a6e0461c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:56.713304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:56.713891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:56.714204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:56.714483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:56.715274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:56.715586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:56.716960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:56.729104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 606/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:56.737706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.835292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66609ec-38b8-4a51-957f-3c5f591c8ebf None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:56.861223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:56.870217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:56.871016 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:56.871322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:56.871601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:56.879735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:56.881190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:10:56.896257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:56.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:10:56.899061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 647/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:56.904698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:56.966656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:57.010428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc548f0-9aa0-42dc-8800-ef148472b781 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:57.044666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:57.045323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:57.045639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:57.045913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:57.046732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:57.047042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:57.048462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:57.065940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 607/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:10:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:57.071807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:57.171895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66fa3b-2712-4cfc-91df-10fa3db550d3 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:57.207519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:57.208161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:57.208449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:57.208732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:57.267574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:57.280096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:57.281661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:57.392019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 648/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:10:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:10:57.398035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:57.507917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2a62-0481-4eca-b8bc-cdb6d249e45c None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:57.522735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:57.546758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:57.547704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:57.547989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:57.548266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:57.559986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:57.561485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:10:57.578999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:57.580381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:57.581433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:10:57.581433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:10:57.581433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:10:57.581433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:57.581433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 608/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:10:57.589800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:57.684773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:57.696533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060144b-03c1-4f8d-af6d-fb21a1a8ebfe None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:57.735936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:57.736722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:57.737042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:57.737322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:57.738092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:57.738398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:57.739791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:57.752706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 649/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:10:57.759391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:57.890284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26215d34-c7b0-42eb-a674-f55e06192de5 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:57.929010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:57.929845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:57.930133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:57.930415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:57.938868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:57.940460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:10:57.955565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:10:57.957811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:10:57.959041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 609/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:10:57.963476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:58.072567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb209-f1ca-4360-b0e4-168d5a94970c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:58.107901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:58.108558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:58.108845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:58.109118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:58.109941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:58.110253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:58.111687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7db2a-c898-4c78-a32a-227be9db28d5 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:58.123801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 650/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:10:58.129946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:10:58.157261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:10:58.230641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eca65-4c3c-4ffc-8453-a84a3861d654 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:10:58.263626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:10:58.264280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:10:58.264555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:10:58.264829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:10:58.265679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ce1f644a909746fe92bd0dee4e9ce635) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:10:58.265993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:10:58.267732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6747e3a-9693-472f-a67f-b0077d95b3cd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:10:58.274422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 610/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:10:58 2022] GET /identity/v3/role_assignments?user.id=ce1f644a909746fe92bd0dee4e9ce635&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:00.467977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15461110-072d-4bfe-be7d-a81a711964a9 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:00.468573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15461110-072d-4bfe-be7d-a81a711964a9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:00.468573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15461110-072d-4bfe-be7d-a81a711964a9 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:00.469973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 651/1261] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:00.475276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:00.475657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:00.475834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:00.494296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:00.495560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:00.564716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 611/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:11:00.586764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15461110-072d-4bfe-be7d-a81a711964a9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:00.598196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:00.693113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15461110-072d-4bfe-be7d-a81a711964a9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:00.729052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:00.729796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:00.730093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:00.730375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:00.740320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:11:00.748489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:00.749711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:00.750698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 652/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:11:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:00.756146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:00.776717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:00.856987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5439d-84cc-4584-8dfd-e5cbf2b19ae2 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:00.891079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:00.900221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:00.904167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 612/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:00.910674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:00.979350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:01.012491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587336f7-1f30-417f-ab32-685cac613de9 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.045794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.046577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.046856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.047131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.058114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.058926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:11:01.076414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:01.077531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:01.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:11:01.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:11:01.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:01.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:01.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 653/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:11:01.083523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.166881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:01.184825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213d94f-ce6a-4171-bbc9-59255ecf2c26 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.218988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.219666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.219965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.220235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.220959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:01.221370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.222870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.236620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 613/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:01.241281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.342549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e30743-f4d5-45f7-8844-f808fe32484a None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.364126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:01.380653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.381705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.382091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.382490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.391389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.393633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:11:01.410425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:01.417018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:01.418073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 654/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:01.419404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.520621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b9226-404f-44b4-8e0d-51a3e3153dcd None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.553890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.554555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.554985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.555150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.556552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:01.556859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.558366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.567804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:01.576297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 614/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:01.581921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.686674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc42554-7698-4882-8534-6be1828cd443 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.720744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.721562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.721857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.722141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.734147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.735679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:11:01.752487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:01.753604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:01.754590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:11:01.754590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:11:01.754590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:01.754590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:01.754590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 655/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:01.759999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.864677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e7263-ed63-4ce2-8982-7a0de979d9c6 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:01.901905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:01.902570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:01.902858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:01.903138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:01.903881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:01.904193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:01.905596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:01.917219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 615/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:01.923071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:01.956025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:02.025059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbac57-cd58-44e7-8ab1-8243ba252cd2 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:02.058224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:02.059037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:02.059321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:02.059594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:02.066880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:02.068383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:11:02.082379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:02.083439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:02.084521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 656/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:11:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:02.090123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:02.155905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:02.196898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be0df7-f349-4543-932b-c9232f984cae None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:02.229921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:02.230578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:02.230864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:02.231144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:02.231981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:02.232269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:02.233774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bf7e2-dff3-463e-a818-496aeea4017d None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:02.246630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 616/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:11:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:02.254734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:02.358410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:02.373527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed45f-b691-4978-97b2-94c868efbc4c None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:02.411841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:02.412651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:02.412936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:02.413214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] PATH_INFO: `/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ce1f644a909746fe92bd0dee4e9ce635/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:02.415561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bb133900b13417cb2458c5ccd8e62d5, user_id=ce1f644a909746fe92bd0dee4e9ce635, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:02.419016 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff603c2f-1a44-4ca9-8fca-fac716299197 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:02.647384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 657/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Fri May 27 16:11:02 2022] PUT /identity/v3/projects/0bb133900b13417cb2458c5ccd8e62d5/users/ce1f644a909746fe92bd0dee4e9ce635/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:11:04.914166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc53b4-7c3c-40af-ad42-a37e771358a4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:04.914697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc53b4-7c3c-40af-ad42-a37e771358a4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:04.914967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc53b4-7c3c-40af-ad42-a37e771358a4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:04.916202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 617/1274] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:04.921696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:04.922172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:04.922260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:04.939442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:04.940644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:05.009551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 658/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:11:05.034937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc53b4-7c3c-40af-ad42-a37e771358a4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.140526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc53b4-7c3c-40af-ad42-a37e771358a4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:05.159956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:05.176612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:05.177333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:05.177650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:05.177931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:05.188570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 16:11:05.197508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:05.198667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:05.199535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 618/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:05.205141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.287172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:05.306118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc118-136d-49a1-b328-11a2ca06a661 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:05.340404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:05.341083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:05.341397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:05.341679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:05.342451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:05.342788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:05.344213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:05.353440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 659/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:05.359386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.473657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba4060-caf2-4806-a7ae-9509add56f67 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:05.519743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:05.521237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:05.521724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:05.522230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:05.541014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:05.543775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:11:05.574563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:05.576781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:05.580115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:11:05.580115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:11:05.580115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:05.580115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:05.580115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 619/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:05.582842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.706496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac8659-904e-4e94-a456-e9ba44f3ca4e None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:05.749394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:05.749980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:05.750267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:05.750547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:05.751273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:05.751584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:05.752970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:05.766540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 660/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:05.772263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.806808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:05.881669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9b8a5-f279-4ca3-a493-28dec609491b None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:05.917930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:05.918730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:05.919013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:05.919351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:05.926961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:05.928403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 16:11:05.943746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:05.947502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:05.948471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 620/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:05.951593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:05.955469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:06.054090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f0d7-2f77-4da4-90db-6089a9eb0e96 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:06.090028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:06.090701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:06.090985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:06.091262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:06.092103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:06.092417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:06.093887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:06.111732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 661/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Fri May 27 16:11:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:06.117456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:06.212955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:06.224683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a66c7-82b9-4f2d-aad4-dc76fc7f4fc8 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:06.264861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:06.266031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:06.266492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:06.266907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:06.347722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:06.349011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:06.449855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:06.460930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 621/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Fri May 27 16:11:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 16:11:06.467611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:06.541576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:06.580205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce697a-8be1-4dfe-8d2a-b394c5771a50 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:06.613315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:06.614126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:06.614410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:06.614685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:06.625753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:06.632266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 16:11:06.644087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:06.645068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:06.646500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 16:11:06.646500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 16:11:06.646500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 16:11:06.646500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:06.646500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 662/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:06.651647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:06.757686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183b30d-521c-4805-9bc8-a426cff91517 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:06.799304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:06.800033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:06.800329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:06.800608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:06.801331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:06.801666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:06.803051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:06.814893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 622/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Fri May 27 16:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:06.821502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:06.927582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50404f-f6c2-4157-859c-faf68c89f603 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:06.942841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:06.966367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:06.967408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:06.967408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:06.967690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:06.974619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:06.976076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 16:11:06.991885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application with Lock( May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:06.992988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: ERROR keystone.server.flask.application May 27 16:11:06.993950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 663/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Fri May 27 16:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:07.000013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:07.051563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:07.101167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce821a-b200-4e27-8c35-e495da64e801 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:07.136233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:07.136897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:07.137186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:07.137503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:07.138389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:07.138705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:07.140155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978cb65-2862-4871-8707-16bb459910d5 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:07.154109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 623/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Fri May 27 16:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:07.160354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:07.266042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581a4c9-c9ea-4fe9-b906-15c9336b5185 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:07.299624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:07.300269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:07.300551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:07.300828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:07.301676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] RBAC: Adding query filter params (role.id=c60a3fdf3263401aa67021ec3e540337, scope.project.id=0bb133900b13417cb2458c5ccd8e62d5, user.id=ce1f644a909746fe92bd0dee4e9ce635) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:07.301990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:07.303420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eab716-f931-4982-a41d-09582e7bf78a None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:07.309304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 664/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Fri May 27 16:11:07 2022] GET /identity/v3/role_assignments?user.id=ce1f644a909746fe92bd0dee4e9ce635&scope.project.id=0bb133900b13417cb2458c5ccd8e62d5&role.id=c60a3fdf3263401aa67021ec3e540337 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:09.552393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f983-df20-4bdc-a342-9e380378c9eb None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:09.552926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f983-df20-4bdc-a342-9e380378c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:09.553056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f983-df20-4bdc-a342-9e380378c9eb None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:09.554392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 624/1288] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:09.560866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:09.561453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:09.561453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:09.578634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:09.579833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:09.609898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:09.646741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 665/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 16:11:09.670119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f983-df20-4bdc-a342-9e380378c9eb None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:09.780585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f983-df20-4bdc-a342-9e380378c9eb None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:09.793188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:09.823096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:09.823096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:09.823096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:09.823096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:09.823684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:09.823684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:09.833714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:09.838423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:09.840525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 625/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:11:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:09.842346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:09.941231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee71036-9cfc-49ac-b151-2ddf8d918f7a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:09.978280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:09.978941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:09.979226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:09.979715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:09.980567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:09.980881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:09.982319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253818-4881-4fae-992f-a934b0843ee4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:09.987407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 666/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Fri May 27 16:11:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:09.993384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:10.100613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f350e-be4b-46f3-bbef-702daaacd1ee None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:10.138018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:10.138900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:10.139194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:10.139504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] PATH_INFO: `/v3/services` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:10.140213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:10.140525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:10.141968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749dbd23-aa2a-4112-9130-6507ccce1351 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:10.147895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 626/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Fri May 27 16:11:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:12.374276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be74fd-d242-4f16-a0fa-0d9e82aee1a2 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:12.375762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be74fd-d242-4f16-a0fa-0d9e82aee1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:12.375762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be74fd-d242-4f16-a0fa-0d9e82aee1a2 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:12.376226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 667/1293] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:12.382562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ad-3027-465d-a394-66db8016680e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:12.383492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ad-3027-465d-a394-66db8016680e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:12.383651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ad-3027-465d-a394-66db8016680e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:12.402762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-efb205ad-3027-465d-a394-66db8016680e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:12.410316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-efb205ad-3027-465d-a394-66db8016680e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:12.434423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:12.478457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 627/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:11:12.502465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be74fd-d242-4f16-a0fa-0d9e82aee1a2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:12.536301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:12.608334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be74fd-d242-4f16-a0fa-0d9e82aee1a2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:12.643195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:12.643778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:12.644059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:12.644331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:12.645038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:12.646466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b27b6a-d5a8-46a7-aa26-52e74a3ce4a1 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:12.765793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 668/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Fri May 27 16:11:12 2022] POST /identity/v3/services => generated 246 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:11:14.993205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1af0a-ead5-4f93-beac-b243eba29636 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:14.993998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1af0a-ead5-4f93-beac-b243eba29636 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:14.994154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1af0a-ead5-4f93-beac-b243eba29636 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:14.995612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 628/1296] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:15.001268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:15.001724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:15.001861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:15.019289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:15.020538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:15.096970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 669/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:11:15.120682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1af0a-ead5-4f93-beac-b243eba29636 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:15.228508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1af0a-ead5-4f93-beac-b243eba29636 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:15.266177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:15.266971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:15.267251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:15.267525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:15.268173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:15.269578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:15.283133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:15.285132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:15.287133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 629/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:11:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 16:11:15.291574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:15.309962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:15.399382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0658b-5de7-427a-b1ce-d23f5147de2c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:15.435130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:15.435847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:15.436136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:15.436469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:15.437248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:15.437604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:15.438999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:15.444577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 670/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Fri May 27 16:11:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:15.451719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:15.486351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:15.557608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb7d74-420b-4423-b0ba-96eaec85fd40 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:15.592425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:15.593089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:15.593529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:15.593843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:15.594639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] RBAC: Adding query filter params (interface=public, service_id=5d89bd42a353444081dfcfc65cd2d329, region_id=RegionOne) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:15.594954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:15.596332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5baa57-c713-4e29-8f49-2f992c191c14 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:15.601570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 630/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Fri May 27 16:11:15 2022] GET /identity/v3/endpoints?service_id=5d89bd42a353444081dfcfc65cd2d329&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:15.607511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:15.721360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbea42-af45-429e-9a53-4f8a15fac399 None admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:15.742596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:15.761505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:15.762177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:15.762471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:15.762749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:15.763474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:15.764912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d983007-7b39-4c73-9aab-c89c4375ecf4 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:15.771818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 671/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Fri May 27 16:11:15 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:11:18.174853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdad1f1-64ef-47e1-a491-680edbdd3db0 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:18.175549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdad1f1-64ef-47e1-a491-680edbdd3db0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:18.175549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdad1f1-64ef-47e1-a491-680edbdd3db0 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:18.176803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 631/1302] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:11:18.183158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:18.183613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:18.183775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:18.201977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:11:18.203189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:11:18.276339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 672/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:11:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 16:11:18.299795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdad1f1-64ef-47e1-a491-680edbdd3db0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:18.408061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:18.416775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdad1f1-64ef-47e1-a491-680edbdd3db0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:18.453220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:18.453971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:18.454267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:18.454570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:18.455229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:18.456602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 16:11:18.468907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 16:11:18.475687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.get_or_create( May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application with Lock( May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self._enter() May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application return self.creator() May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application created_value = creator( May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: ERROR keystone.server.flask.application May 27 16:11:18.477408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 632/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Fri May 27 16:11:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 16:11:18.478366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:18.553413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:18.576836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6ceaa-7e49-4e9f-969c-486d93a589bf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:18.610599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:18.611264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:18.611578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:18.611830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] PATH_INFO: `/v3/services` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:18.612554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:11:18.612845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:18.614249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd8ac7-cd84-4f39-923b-9fa446b75e85 None admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:18.621170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 673/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Fri May 27 16:11:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:11:18.626954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:11:18.730058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7a184-8e88-4ab4-aa24-9ec23f865957 None admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:11:18.763350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:11:18.764005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:11:18.764290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:11:18.764569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:11:18.765309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:11:18.766717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599294-85bf-4749-9c09-911b4b9890ba None admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:11:18.895829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 633/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Fri May 27 16:11:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:11:28.619540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=30 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:11:28.890977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:32.411154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fad9-7803-4ac1-b6fc-7c645b59da39 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:32.411154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fad9-7803-4ac1-b6fc-7c645b59da39 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:32.411154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580fad9-7803-4ac1-b6fc-7c645b59da39 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:32.416341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 674/1307] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:32.419067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d95144-8c2f-4c47-924c-497fafc1e7cd None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:32.419471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d95144-8c2f-4c47-924c-497fafc1e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:32.419887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d95144-8c2f-4c47-924c-497fafc1e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:32.622518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-99d95144-8c2f-4c47-924c-497fafc1e7cd None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:32.626900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-99d95144-8c2f-4c47-924c-497fafc1e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:32.769153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 634/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:32.830438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df496be-439b-4927-8913-081a4081bd4e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:32.830772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df496be-439b-4927-8913-081a4081bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:32.831503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df496be-439b-4927-8913-081a4081bd4e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:32.832980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 675/1309] 10.222.0.42 () {54 vars in 964 bytes} [Fri May 27 16:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:32.839928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9cfa1-8442-4e5d-bb0e-95aca973312d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:32.840254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9cfa1-8442-4e5d-bb0e-95aca973312d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:32.840520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9cfa1-8442-4e5d-bb0e-95aca973312d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:32.939198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c0e9cfa1-8442-4e5d-bb0e-95aca973312d None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:32.940960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c0e9cfa1-8442-4e5d-bb0e-95aca973312d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:32.968978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:33.032223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 635/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Fri May 27 16:19:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:33.041753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df496be-439b-4927-8913-081a4081bd4e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:33.043534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9df496be-439b-4927-8913-081a4081bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:33.180659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df496be-439b-4927-8913-081a4081bd4e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:33.282090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:33.283232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:33.283739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:33.284785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:33.286931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:33.353543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:33.418285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:33.419775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341bd6a-3bf5-4d96-9541-cc41a068b1b1 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:33.440806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 676/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:36.180350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:36.181147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:36.181147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:36.182758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 636/1312] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:19:36.187705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743043e-fbaf-4ecf-8f7b-58512049ac83 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:36.188049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743043e-fbaf-4ecf-8f7b-58512049ac83 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:36.188253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743043e-fbaf-4ecf-8f7b-58512049ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:36.282511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f743043e-fbaf-4ecf-8f7b-58512049ac83 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:36.283894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f743043e-fbaf-4ecf-8f7b-58512049ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:36.428833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 677/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:36.467429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:36.469221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:36.581479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37987eb-08bd-400b-a7b9-7d0e8fef458f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:36.673463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:36.676927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:36.677879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:36.678164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:36.678445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:36.680174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:36.811305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:36.812932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe60962-5cd7-44ad-8075-de57b32cf88d service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:36.846475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 637/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:39.665807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20eb1-bfca-40ad-a7f3-616fb95de220 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:39.666404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20eb1-bfca-40ad-a7f3-616fb95de220 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:39.666404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20eb1-bfca-40ad-a7f3-616fb95de220 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:39.667723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 678/1315] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:39.672929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755c7-f4d6-40ca-9d24-2c5eaf1eef50 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:39.673399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755c7-f4d6-40ca-9d24-2c5eaf1eef50 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:39.673530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4755c7-f4d6-40ca-9d24-2c5eaf1eef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:39.739706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3a4755c7-f4d6-40ca-9d24-2c5eaf1eef50 None None] MFA Rules not processed for user `460698a7b6d64938b66f2e216b2e86c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:39.741201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3a4755c7-f4d6-40ca-9d24-2c5eaf1eef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:39.806678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:39.849927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 638/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:19:39.906925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:39.907210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:39.907489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:39.909035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 679/1317] 10.222.0.42 () {54 vars in 964 bytes} [Fri May 27 16:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:39.915100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff8dc-936d-450e-a973-522a78e4fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:39.915392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff8dc-936d-450e-a973-522a78e4fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:39.915671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff8dc-936d-450e-a973-522a78e4fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:39.991860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3ceff8dc-936d-450e-a973-522a78e4fcc2 None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:39.993050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3ceff8dc-936d-450e-a973-522a78e4fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:40.073441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 639/1318] 10.222.0.42 () {58 vars in 1061 bytes} [Fri May 27 16:19:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:40.082786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:40.084331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:40.094170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:40.204781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170536b-0b46-4a08-8593-bd799cf28b80 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:40.282864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:40.283718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:40.284007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:40.284286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:40.285957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:40.351991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:40.400826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:40.402401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af317d3d-c557-454e-8a8e-1899115de6e6 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:40.421604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 680/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:43.929564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:43.930116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:43.930116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:43.931487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 640/1320] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:19:43.938076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140895d3-c368-4b2a-8ca6-aae8c82be963 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:43.938524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140895d3-c368-4b2a-8ca6-aae8c82be963 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:43.938648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140895d3-c368-4b2a-8ca6-aae8c82be963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:44.009572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-140895d3-c368-4b2a-8ca6-aae8c82be963 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:44.010793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-140895d3-c368-4b2a-8ca6-aae8c82be963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:44.115647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 681/1321] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:44.154996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:44.155401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:44.209771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:44.269707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef92b-7d96-4618-a829-b7b8fcb74c23 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:44.343430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:44.344065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:44.344347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:44.344623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:44.346779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:44.445652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:44.458949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:44.460446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd28971-b369-4c25-8d0f-8917076e25dc service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:44.478431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 641/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:47.941080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:47.941661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:47.941661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:47.943184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 682/1323] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:47.948396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c532454-a354-4534-b8cd-78ede80b5e9f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:47.951305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c532454-a354-4534-b8cd-78ede80b5e9f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:47.951305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c532454-a354-4534-b8cd-78ede80b5e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:48.026728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1c532454-a354-4534-b8cd-78ede80b5e9f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:48.028465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1c532454-a354-4534-b8cd-78ede80b5e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:48.153667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 642/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:19:48.188465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:48.190799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:48.210350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:48.297699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3bfcf-a3f0-46e5-bd40-973a51b174b8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:48.377856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:48.378525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:48.378817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:48.379099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:48.380778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:48.461225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:48.503341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:48.504793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb752276-cd13-4d84-b4f7-06eb7d6ac797 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:48.524117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 683/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:50.860867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:50.861978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:50.862446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:50.864206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 643/1326] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:50.870100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742572-26ea-47ae-8235-2e0b0f0d197a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:50.870584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742572-26ea-47ae-8235-2e0b0f0d197a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:50.870786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff742572-26ea-47ae-8235-2e0b0f0d197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:50.944050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ff742572-26ea-47ae-8235-2e0b0f0d197a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:50.945303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ff742572-26ea-47ae-8235-2e0b0f0d197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:51.047216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 684/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:51.084406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:51.085960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:51.161183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:51.215143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7e35-2ab2-4583-8be6-c5c90ff29e01 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:51.294068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:51.294732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:51.295033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:51.295308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:51.296942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:51.407286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:51.411002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:51.412585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e49978-98e1-4a90-974d-de6b2a5ffb1c service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:51.431331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 644/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:54.095015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:54.095589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:54.095589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:54.097097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 685/1329] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:54.102895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed091d7-5dc5-4617-b0c0-55cc986720eb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:54.103378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed091d7-5dc5-4617-b0c0-55cc986720eb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:54.103546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed091d7-5dc5-4617-b0c0-55cc986720eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:54.174757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2ed091d7-5dc5-4617-b0c0-55cc986720eb None None] MFA Rules not processed for user `460698a7b6d64938b66f2e216b2e86c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:54.176178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2ed091d7-5dc5-4617-b0c0-55cc986720eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:54.285650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 645/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:54.327874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:54.329655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:54.367377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:54.451616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1c4ee-74b3-4cc1-83f5-1ee1020d26a4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:54.529262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:54.529953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:54.530243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:54.530520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:54.532214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:54.618108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:54.651129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:54.653023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef831b4f-fe57-449d-aa27-289955258dc2 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:54.672704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 686/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:19:57.679561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:57.680162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:57.680162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:57.681594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 646/1332] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:19:57.686957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9afb-e723-4c5d-af2a-448bed196e88 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:57.687399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9afb-e723-4c5d-af2a-448bed196e88 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:57.687589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9afb-e723-4c5d-af2a-448bed196e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:57.773390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-badc9afb-e723-4c5d-af2a-448bed196e88 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:19:57.774628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-badc9afb-e723-4c5d-af2a-448bed196e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:57.894603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 687/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:19:57.931599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:19:57.933243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:58.011365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:58.054215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e7e99-e26c-42ae-b98e-1d4b17c7994e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:19:58.136854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:19:58.137587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:19:58.137879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:19:58.138154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:19:58.139799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:19:58.256090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:19:58.257459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1473e-fb02-467b-8ffa-b5866e8e706a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:19:58.267666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:19:58.277310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 647/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:00.574186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:00.574186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:00.574186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:00.574907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 688/1335] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:00.580894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5e2f-18e6-453e-afec-4009a170bd65 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:00.581381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5e2f-18e6-453e-afec-4009a170bd65 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:00.581507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5e2f-18e6-453e-afec-4009a170bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:00.647764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a77a5e2f-18e6-453e-afec-4009a170bd65 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:00.648914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a77a5e2f-18e6-453e-afec-4009a170bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:00.745406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 648/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:00.785315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:00.786916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:00.827316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:00.893007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0b7e0-5136-4b36-93ec-ef9fb541cd24 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:00.966993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:00.967637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:00.967919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:00.968195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:00.969912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:01.054768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:01.079858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:01.081295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2241bd-0279-47df-8e97-d50e18eb0e9b service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:01.102089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 689/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:04.043776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:04.044330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:04.044330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:04.045859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 649/1338] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:04.051960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05d61-9e6d-4f60-964c-8303b9c91829 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:04.052373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05d61-9e6d-4f60-964c-8303b9c91829 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:04.053662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a05d61-9e6d-4f60-964c-8303b9c91829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:04.123707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-53a05d61-9e6d-4f60-964c-8303b9c91829 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:04.124964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-53a05d61-9e6d-4f60-964c-8303b9c91829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:04.237734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 690/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:04.273501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:04.275710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:04.358676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:04.373606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123f8e-0c0a-4cf2-80d4-afb0c0551e5e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:04.378180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:04.378825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:04.379108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:04.379381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:04.388304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:04.391142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505702-1e59-4f3a-867c-1f08439c2294 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:04.399660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 650/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:08.995695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810a344-8483-44ed-8a11-4114d180b136 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:08.996264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810a344-8483-44ed-8a11-4114d180b136 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:08.996264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810a344-8483-44ed-8a11-4114d180b136 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:08.997691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 691/1341] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:09.003638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b1ae-b3a2-4a45-bc50-e0e187d2cbcf None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:09.003638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b1ae-b3a2-4a45-bc50-e0e187d2cbcf None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:09.003638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b1ae-b3a2-4a45-bc50-e0e187d2cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:09.035522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d716b1ae-b3a2-4a45-bc50-e0e187d2cbcf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:09.036750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d716b1ae-b3a2-4a45-bc50-e0e187d2cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:09.133046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 651/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:09.171758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810a344-8483-44ed-8a11-4114d180b136 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:09.182863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810a344-8483-44ed-8a11-4114d180b136 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:09.186380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:09.187021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:09.187310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:09.188112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:09.197516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:09.198902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747399f-b367-4b5f-af4b-abdc5eb9a529 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:09.210893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 692/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:20:12.630552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:12.631108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:12.631108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:12.632513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 652/1344] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:12.638149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87840-91c4-414c-9f66-4d373409beba None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:12.642122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87840-91c4-414c-9f66-4d373409beba None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:12.642122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f87840-91c4-414c-9f66-4d373409beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:12.661327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b1f87840-91c4-414c-9f66-4d373409beba None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:12.662551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b1f87840-91c4-414c-9f66-4d373409beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:12.670718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 693/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:12.708570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:12.710467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:12.726123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e1bc3-fdfd-496e-ba72-0c5931518a54 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:12.729714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:12.730365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:12.730674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:12.730952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:12.741031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:12.742523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843a3d7-edaf-45d5-8358-359763d61947 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:12.753277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 653/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:17.643174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1349-de8a-4cf4-8af8-2443f270ba34 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:17.643728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1349-de8a-4cf4-8af8-2443f270ba34 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:17.643728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff1349-de8a-4cf4-8af8-2443f270ba34 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:17.645094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 694/1347] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:20:17.650906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab284177-8439-4564-9915-65c57219ac3c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:17.651535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab284177-8439-4564-9915-65c57219ac3c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:17.651535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab284177-8439-4564-9915-65c57219ac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:17.670003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ab284177-8439-4564-9915-65c57219ac3c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:17.671234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ab284177-8439-4564-9915-65c57219ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:17.680224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 654/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:17.718967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1349-de8a-4cf4-8af8-2443f270ba34 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:17.730574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff1349-de8a-4cf4-8af8-2443f270ba34 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:17.735295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:17.735952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:17.736308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:17.736596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:17.747057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:17.748592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb86cb6-5127-42c9-877b-be71b2cd24db service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:17.760769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 695/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:23.303928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c406-999b-48a4-a943-2845a23f1969 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:23.304896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c406-999b-48a4-a943-2845a23f1969 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:23.305430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c406-999b-48a4-a943-2845a23f1969 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:23.307516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 655/1350] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:20:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:23.321503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78152-57b5-4e77-970f-97cd659ea145 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:23.322450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78152-57b5-4e77-970f-97cd659ea145 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:23.322946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78152-57b5-4e77-970f-97cd659ea145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:23.380011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:23.442228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1fe78152-57b5-4e77-970f-97cd659ea145 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:23.443778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1fe78152-57b5-4e77-970f-97cd659ea145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:23.520342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 696/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:20:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:27.665950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=57 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:28.143363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4efde-6005-488e-9308-a933272fb819 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:28.143807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4efde-6005-488e-9308-a933272fb819 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:28.143929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4efde-6005-488e-9308-a933272fb819 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:28.145491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 656/1352] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:28.150725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2907-4413-47da-8744-85df1dd97ed0 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:28.151234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2907-4413-47da-8744-85df1dd97ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:28.151320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2907-4413-47da-8744-85df1dd97ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:28.173103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fb3a2907-4413-47da-8744-85df1dd97ed0 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:28.174362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fb3a2907-4413-47da-8744-85df1dd97ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:28.183282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 697/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:20:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:29.582051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a6cd-6b38-4bbe-afcf-e13ef8ca1041 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:29.582633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a6cd-6b38-4bbe-afcf-e13ef8ca1041 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:29.582633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a6cd-6b38-4bbe-afcf-e13ef8ca1041 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:29.584042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 657/1354] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:29.589642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb0b8-de94-41e4-afa1-efe2ac9366da None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:29.590076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb0b8-de94-41e4-afa1-efe2ac9366da None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:29.590371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bb0b8-de94-41e4-afa1-efe2ac9366da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:29.612637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-251bb0b8-de94-41e4-afa1-efe2ac9366da None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:29.613984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-251bb0b8-de94-41e4-afa1-efe2ac9366da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:29.623291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 698/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:20:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:33.443050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187b3f-d90e-4884-a119-eb69fb83f8cd None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:33.444345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187b3f-d90e-4884-a119-eb69fb83f8cd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:33.445094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187b3f-d90e-4884-a119-eb69fb83f8cd None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:33.447429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 658/1356] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:33.453556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4f8b9-a541-4f74-8069-61ec7048d94d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:33.454635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4f8b9-a541-4f74-8069-61ec7048d94d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:33.455259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4f8b9-a541-4f74-8069-61ec7048d94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:33.476228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9fb4f8b9-a541-4f74-8069-61ec7048d94d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:33.477631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9fb4f8b9-a541-4f74-8069-61ec7048d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:33.487639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 699/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:33.545128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d0c3-94eb-435b-bcf6-522949eb059b None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:33.546301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d0c3-94eb-435b-bcf6-522949eb059b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:33.546784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78d0c3-94eb-435b-bcf6-522949eb059b None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:33.548512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 659/1358] 10.222.0.42 () {54 vars in 961 bytes} [Fri May 27 16:20:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:33.561172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b230-6ec2-4a38-82af-95e05a87df85 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:33.561882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b230-6ec2-4a38-82af-95e05a87df85 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:33.562510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64b230-6ec2-4a38-82af-95e05a87df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:33.621607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2f64b230-6ec2-4a38-82af-95e05a87df85 None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:33.622860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2f64b230-6ec2-4a38-82af-95e05a87df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:33.694512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:33.719178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 700/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Fri May 27 16:20:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:33.728937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78d0c3-94eb-435b-bcf6-522949eb059b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:33.745605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78d0c3-94eb-435b-bcf6-522949eb059b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:33.755563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:33.756401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:33.756856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:33.757322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:33.767572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:33.769267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99ff49-0538-4c76-8820-57f38c5031e1 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:33.780202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 660/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:36.305152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d1ce-78ee-484b-b1fc-7110472ffe21 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:36.305700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d1ce-78ee-484b-b1fc-7110472ffe21 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:36.305700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6d1ce-78ee-484b-b1fc-7110472ffe21 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:36.307018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 701/1361] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:36.313926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04920-a772-4ada-8841-a2a23fa49ab4 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:36.314523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04920-a772-4ada-8841-a2a23fa49ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:36.314709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04920-a772-4ada-8841-a2a23fa49ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:36.333806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-26b04920-a772-4ada-8841-a2a23fa49ab4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:36.335044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-26b04920-a772-4ada-8841-a2a23fa49ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:36.343542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 661/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:36.392750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6d1ce-78ee-484b-b1fc-7110472ffe21 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:36.407966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6d1ce-78ee-484b-b1fc-7110472ffe21 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:36.416754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:36.417443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:36.417741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:36.418037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:36.428639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:36.430291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4cf9b-8539-4cb4-aa76-438541c22ca5 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:36.443061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 702/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:20:39.286422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf91-c653-4769-8460-ffa558791792 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:39.286977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf91-c653-4769-8460-ffa558791792 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:39.286977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adf91-c653-4769-8460-ffa558791792 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:39.288408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 662/1364] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:39.294085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79244bd-e320-4d60-a6ad-e136d7cfa6ca None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:39.294963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79244bd-e320-4d60-a6ad-e136d7cfa6ca None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:39.295411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79244bd-e320-4d60-a6ad-e136d7cfa6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:39.324203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a79244bd-e320-4d60-a6ad-e136d7cfa6ca None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:39.324868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a79244bd-e320-4d60-a6ad-e136d7cfa6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:39.337272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 703/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:39.393609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2199-9cf3-480d-95fb-012723aacbac None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:39.394047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2199-9cf3-480d-95fb-012723aacbac None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:39.394443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e2199-9cf3-480d-95fb-012723aacbac None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:39.395839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 663/1366] 10.222.0.42 () {54 vars in 961 bytes} [Fri May 27 16:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:39.402649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e374-51a2-4344-95c8-84aa3081e99b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:39.402940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e374-51a2-4344-95c8-84aa3081e99b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:39.403224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e374-51a2-4344-95c8-84aa3081e99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:39.425849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9097e374-51a2-4344-95c8-84aa3081e99b None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:39.427239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9097e374-51a2-4344-95c8-84aa3081e99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:39.446153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 704/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Fri May 27 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:39.454988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2199-9cf3-480d-95fb-012723aacbac None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:39.469684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e2199-9cf3-480d-95fb-012723aacbac None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:39.475212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:39.475852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:39.476134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:39.476409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:39.485505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:39.487578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7e578-76a1-4c35-82f2-de2220f1217e service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:39.497053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 664/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:41.992778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea343b9-39a4-4b9f-a27a-4c36d192c518 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:41.993484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea343b9-39a4-4b9f-a27a-4c36d192c518 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:41.993484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea343b9-39a4-4b9f-a27a-4c36d192c518 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:41.994989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 705/1369] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:42.001427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e975c-5851-41e7-9837-39d624bd31b7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.002297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e975c-5851-41e7-9837-39d624bd31b7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.002804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e975c-5851-41e7-9837-39d624bd31b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.022952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1d7e975c-5851-41e7-9837-39d624bd31b7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:42.024299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1d7e975c-5851-41e7-9837-39d624bd31b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:42.034830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 665/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:42.088601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea343b9-39a4-4b9f-a27a-4c36d192c518 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:42.105311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea343b9-39a4-4b9f-a27a-4c36d192c518 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:42.112595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:42.113477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.113827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.114178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.124482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:42.126297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad30d95-1cc0-4b54-82f9-be51ec001b60 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:42.137889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 706/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:42.214514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3a8c-3fba-425b-b0d8-8fdaba38b4a4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.214720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3a8c-3fba-425b-b0d8-8fdaba38b4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.215048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a3a8c-3fba-425b-b0d8-8fdaba38b4a4 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.217283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 666/1372] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:42.224014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280971bc-665e-48b0-917e-ad231d9bfd40 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.224313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280971bc-665e-48b0-917e-ad231d9bfd40 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.224616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280971bc-665e-48b0-917e-ad231d9bfd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.243742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-280971bc-665e-48b0-917e-ad231d9bfd40 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:42.244949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-280971bc-665e-48b0-917e-ad231d9bfd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:42.253915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 707/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:42.559068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252650-1d80-46a5-b434-f66f2a2d3873 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.564520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252650-1d80-46a5-b434-f66f2a2d3873 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.564520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17252650-1d80-46a5-b434-f66f2a2d3873 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.564520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 667/1374] 10.222.0.42 () {54 vars in 963 bytes} [Fri May 27 16:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:42.569049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a895-9789-47c9-a67e-db31e0d8786b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.569534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a895-9789-47c9-a67e-db31e0d8786b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.570053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75a895-9789-47c9-a67e-db31e0d8786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.657415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2a75a895-9789-47c9-a67e-db31e0d8786b None None] MFA Rules not processed for user `2be41ec2de3948e6a015dfe35c47a0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:42.659150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2a75a895-9789-47c9-a67e-db31e0d8786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:42.748909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 708/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Fri May 27 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:42.757791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252650-1d80-46a5-b434-f66f2a2d3873 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:42.772582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17252650-1d80-46a5-b434-f66f2a2d3873 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:42.780166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:42.780166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:42.780166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:42.780533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:42.794064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:42.797620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa909-8deb-4e34-85cd-e7b51dc2e2a7 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:42.809942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 668/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:44.762476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0b54-3bfa-467f-98e0-9ce44f07be2b None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:44.763254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0b54-3bfa-467f-98e0-9ce44f07be2b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:44.763254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0b54-3bfa-467f-98e0-9ce44f07be2b None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:44.764623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 709/1377] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:44.770456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014dd1-a0e5-48ac-98dd-902ca9def47a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:44.770958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014dd1-a0e5-48ac-98dd-902ca9def47a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:44.771056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53014dd1-a0e5-48ac-98dd-902ca9def47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:44.789927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-53014dd1-a0e5-48ac-98dd-902ca9def47a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:44.791340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-53014dd1-a0e5-48ac-98dd-902ca9def47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:44.800682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 669/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:44.845982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a0b54-3bfa-467f-98e0-9ce44f07be2b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:44.864369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a0b54-3bfa-467f-98e0-9ce44f07be2b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:44.870009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:44.870665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:44.870960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:44.871249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:44.881856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:44.883483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82113981-da33-4925-8ae7-17e0111090ef service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:44.894926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 710/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:47.346052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40de8-d9a8-4d52-ac97-2a20f5b24432 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:47.346614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40de8-d9a8-4d52-ac97-2a20f5b24432 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:47.346614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c40de8-d9a8-4d52-ac97-2a20f5b24432 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:47.347902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 670/1380] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:47.353141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30661a79-a9f5-4164-ab5a-6ff5c47eeb35 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:47.353618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30661a79-a9f5-4164-ab5a-6ff5c47eeb35 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:47.353733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30661a79-a9f5-4164-ab5a-6ff5c47eeb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:47.371736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-30661a79-a9f5-4164-ab5a-6ff5c47eeb35 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:47.373519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-30661a79-a9f5-4164-ab5a-6ff5c47eeb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:47.382996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 711/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:47.429870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c40de8-d9a8-4d52-ac97-2a20f5b24432 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:47.445067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c40de8-d9a8-4d52-ac97-2a20f5b24432 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:47.451892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:47.452510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:47.452977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:47.453546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:47.462316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:47.464463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caf0d7-8b9c-4431-b694-3b79216dd9cf service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:47.475451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 671/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:49.947734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66446a3-85bf-4a88-b776-e7910194c0d2 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:49.948800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66446a3-85bf-4a88-b776-e7910194c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:49.949150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66446a3-85bf-4a88-b776-e7910194c0d2 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:49.950726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 712/1383] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:49.957072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817547c3-ec49-4e38-82e8-94e421b74d1c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:49.957776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817547c3-ec49-4e38-82e8-94e421b74d1c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:49.957776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817547c3-ec49-4e38-82e8-94e421b74d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:49.979341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-817547c3-ec49-4e38-82e8-94e421b74d1c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:49.980790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-817547c3-ec49-4e38-82e8-94e421b74d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:49.990017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 672/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:50.035979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66446a3-85bf-4a88-b776-e7910194c0d2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:50.051403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66446a3-85bf-4a88-b776-e7910194c0d2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:50.056801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:50.057466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:50.057752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:50.058026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:50.067815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:50.069723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934bd10-b037-4c7c-879e-a87139b35c04 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:50.081075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 713/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:52.382086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fce11-d430-49db-a2d4-6c37f9c3fda7 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:52.382629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fce11-d430-49db-a2d4-6c37f9c3fda7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:52.382629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fce11-d430-49db-a2d4-6c37f9c3fda7 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:52.384103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 673/1386] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:52.390885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba8e-dbb8-4742-a88a-6b2c30caaac5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:52.390885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba8e-dbb8-4742-a88a-6b2c30caaac5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:52.391829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cba8e-dbb8-4742-a88a-6b2c30caaac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:52.409753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5a6cba8e-dbb8-4742-a88a-6b2c30caaac5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:52.411044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5a6cba8e-dbb8-4742-a88a-6b2c30caaac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:52.420115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 714/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:52.465858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fce11-d430-49db-a2d4-6c37f9c3fda7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:52.487479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fce11-d430-49db-a2d4-6c37f9c3fda7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:52.495175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:52.495946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:52.496266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:52.496558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:52.507539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:52.509720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc90ea-06ae-4695-80eb-a79167a91c3a service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:52.523287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 674/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:55.057296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18de7-a2c8-4389-8db9-b389d192c665 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:55.058054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18de7-a2c8-4389-8db9-b389d192c665 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:55.058707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18de7-a2c8-4389-8db9-b389d192c665 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:55.061933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 715/1389] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:55.069158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a9aa-6618-4127-9d6e-cf71c47b1729 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:55.069856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a9aa-6618-4127-9d6e-cf71c47b1729 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:55.070464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24a9aa-6618-4127-9d6e-cf71c47b1729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:55.094116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0c24a9aa-6618-4127-9d6e-cf71c47b1729 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:55.095402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0c24a9aa-6618-4127-9d6e-cf71c47b1729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:55.103832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 675/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:20:55.157915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad18de7-a2c8-4389-8db9-b389d192c665 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:55.176749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad18de7-a2c8-4389-8db9-b389d192c665 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:55.182295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:55.183008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:55.183414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:55.183705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:55.199252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:55.208254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac07dc-6f3b-48e9-a536-091ef4409a6c service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:55.214560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 716/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:57.962597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630b11-55db-4b40-84a6-d2ba7bf1b06e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:57.963406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630b11-55db-4b40-84a6-d2ba7bf1b06e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:57.963568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50630b11-55db-4b40-84a6-d2ba7bf1b06e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:57.967232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 676/1392] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:57.976198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41aeb-698f-4f19-a841-1ba0e882cef2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:57.977050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41aeb-698f-4f19-a841-1ba0e882cef2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:57.977050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41aeb-698f-4f19-a841-1ba0e882cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:58.003998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f7c41aeb-698f-4f19-a841-1ba0e882cef2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:58.005653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f7c41aeb-698f-4f19-a841-1ba0e882cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:58.016872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 717/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:58.069313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630b11-55db-4b40-84a6-d2ba7bf1b06e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:58.079391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=46 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:20:58.091814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50630b11-55db-4b40-84a6-d2ba7bf1b06e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:58.102384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:58.103508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:58.104011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:58.104570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:58.119504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:58.121221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f37a9f-9e0d-4cf7-8b77-a1eff88eaa18 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:58.133225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 677/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:20:59.553576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b612-553a-469e-a7f4-5325eee47974 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:59.554153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b612-553a-469e-a7f4-5325eee47974 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:59.554153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b612-553a-469e-a7f4-5325eee47974 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:59.555534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 718/1395] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:59.562006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29776c74-ecc2-48d6-b777-de52f203eed2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:59.562826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29776c74-ecc2-48d6-b777-de52f203eed2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:59.562826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29776c74-ecc2-48d6-b777-de52f203eed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:59.581451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-29776c74-ecc2-48d6-b777-de52f203eed2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:59.582715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-29776c74-ecc2-48d6-b777-de52f203eed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:59.591195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 678/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:59.652913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dcbea-86c5-48bf-bf9b-f5bfb501b96c None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:59.653251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dcbea-86c5-48bf-bf9b-f5bfb501b96c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:59.653598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dcbea-86c5-48bf-bf9b-f5bfb501b96c None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:59.654939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 719/1397] 10.222.0.42 () {54 vars in 962 bytes} [Fri May 27 16:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:20:59.660698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5219-8684-4c6a-8837-b04a1fc46750 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:59.661194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5219-8684-4c6a-8837-b04a1fc46750 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:59.661705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5219-8684-4c6a-8837-b04a1fc46750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:59.719855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-89fe5219-8684-4c6a-8837-b04a1fc46750 None None] MFA Rules not processed for user `e7017c7303f94b6380b99cb0b09d7e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:20:59.721069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-89fe5219-8684-4c6a-8837-b04a1fc46750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:20:59.785925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 679/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Fri May 27 16:20:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:20:59.793674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dcbea-86c5-48bf-bf9b-f5bfb501b96c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:20:59.805240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dcbea-86c5-48bf-bf9b-f5bfb501b96c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:20:59.809632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:20:59.810421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:20:59.810774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:20:59.811129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:20:59.821010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:20:59.822532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e41e80-36aa-4675-8636-b80e1e16ecf8 service glance] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:20:59.833763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 720/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:00.582414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b573aa-9479-45b0-a09c-9425bec4b584 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.583117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b573aa-9479-45b0-a09c-9425bec4b584 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.583117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b573aa-9479-45b0-a09c-9425bec4b584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.636466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-47b573aa-9479-45b0-a09c-9425bec4b584 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:00.638237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-47b573aa-9479-45b0-a09c-9425bec4b584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:00.714036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 680/1400] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:21:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:00.726852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a808-3118-4132-9489-9dcbfbf4adfe None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.727834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a808-3118-4132-9489-9dcbfbf4adfe None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.729022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3a808-3118-4132-9489-9dcbfbf4adfe None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.729972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 721/1401] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:00.736618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.736923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.737236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.757251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:00.758527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:00.763731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c92b9-973b-4ef3-a675-679c49698b42 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.764018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c92b9-973b-4ef3-a675-679c49698b42 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.764303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c92b9-973b-4ef3-a675-679c49698b42 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.765699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 722/1402] 10.222.0.42 () {54 vars in 942 bytes} [Fri May 27 16:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:00.768100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 681/1403] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:00.772168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2e55-c36b-42cc-8183-e858c7c89306 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.772462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2e55-c36b-42cc-8183-e858c7c89306 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.772743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a2e55-c36b-42cc-8183-e858c7c89306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.787161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:00.814149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:00.825516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-340a2e55-c36b-42cc-8183-e858c7c89306 None None] MFA Rules not processed for user `0c95e2d016a84b5a804f4d18816a81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:00.827773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-340a2e55-c36b-42cc-8183-e858c7c89306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:00.828146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4c4a-dd5c-4805-b64c-645423908ca6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:00.833862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:00.834511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.834801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.835079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.844537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:00.846185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:00.855894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 682/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:21:00.902304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 723/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Fri May 27 16:21:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:00.912612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:00.924388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d1ff9-0cab-455b-884c-6baf048fca72 service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:00.930646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:00.931297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:00.931582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:00.931867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:00.940685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:00.942391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bfa5c-3354-44bf-8a20-42d53df54ca3 service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:00.946187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:00.952107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 683/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:21:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:21:03.565620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a038-625d-43e5-a621-cc09f16e77d5 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:03.566190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a038-625d-43e5-a621-cc09f16e77d5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:03.566190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6a038-625d-43e5-a621-cc09f16e77d5 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:03.567609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 724/1407] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:03.573918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23125f9-00c6-4d3f-85c7-910f81008ef7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:03.575072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23125f9-00c6-4d3f-85c7-910f81008ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:03.576047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23125f9-00c6-4d3f-85c7-910f81008ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:03.599898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a23125f9-00c6-4d3f-85c7-910f81008ef7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:03.601301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a23125f9-00c6-4d3f-85c7-910f81008ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:03.610870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 684/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:03.663633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6a038-625d-43e5-a621-cc09f16e77d5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:03.684930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6a038-625d-43e5-a621-cc09f16e77d5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:03.696576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:03.696576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:03.696985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:03.698213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:03.708022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:03.711679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ea0ee-beb4-40f7-a1e7-5a28ea83925f service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:03.726109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 725/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:13.589850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:14.138288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=47 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:26.721536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf740c-6fa1-4952-857d-8eab24d6a837 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:26.722134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf740c-6fa1-4952-857d-8eab24d6a837 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:26.722134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf740c-6fa1-4952-857d-8eab24d6a837 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:26.723550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 685/1410] 10.222.0.42 () {54 vars in 918 bytes} [Fri May 27 16:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:26.730089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67967828-14b0-47c6-935b-8a8646026941 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:26.730594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67967828-14b0-47c6-935b-8a8646026941 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:26.730702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67967828-14b0-47c6-935b-8a8646026941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:26.749320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-67967828-14b0-47c6-935b-8a8646026941 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:26.750612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-67967828-14b0-47c6-935b-8a8646026941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:26.759257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 726/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Fri May 27 16:21:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:26.775369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec512-2649-4910-b521-4cf90bb5b13e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:26.776249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec512-2649-4910-b521-4cf90bb5b13e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:26.776592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaec512-2649-4910-b521-4cf90bb5b13e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:26.778035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 686/1412] 10.222.0.42 () {54 vars in 918 bytes} [Fri May 27 16:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:26.783070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e586-1834-49eb-9cd8-5f62e6f5099c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:26.783428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e586-1834-49eb-9cd8-5f62e6f5099c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:26.783711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4e586-1834-49eb-9cd8-5f62e6f5099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:26.804262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-8ce4e586-1834-49eb-9cd8-5f62e6f5099c None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:26.804885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8ce4e586-1834-49eb-9cd8-5f62e6f5099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:26.815110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 727/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Fri May 27 16:21:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:29.580834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ebb1-d4c4-4899-aaf1-c34cbde6937d None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:29.581376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ebb1-d4c4-4899-aaf1-c34cbde6937d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:29.581376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ebb1-d4c4-4899-aaf1-c34cbde6937d None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:29.582678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 687/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:29.588706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb385aa-33f6-4427-b6e7-e032e9bc0784 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:29.589190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb385aa-33f6-4427-b6e7-e032e9bc0784 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:29.589299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb385aa-33f6-4427-b6e7-e032e9bc0784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:29.606711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ddb385aa-33f6-4427-b6e7-e032e9bc0784 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:29.608189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ddb385aa-33f6-4427-b6e7-e032e9bc0784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:29.616104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 728/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:29.661412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ebb1-d4c4-4899-aaf1-c34cbde6937d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:29.676650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ebb1-d4c4-4899-aaf1-c34cbde6937d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:29.684713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:29.685379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:29.685670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:29.685952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:29.694481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:29.696075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fff42b-5f4c-4757-8dfe-bbc6492580df service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:29.706827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 688/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:30.701550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1182d-f898-4752-809a-5a15baa0d630 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:30.702079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1182d-f898-4752-809a-5a15baa0d630 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:30.702079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a1182d-f898-4752-809a-5a15baa0d630 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:30.703390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 729/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:30.708415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ff861-81cd-41bf-882a-26ed7e6b65b6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:30.708847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ff861-81cd-41bf-882a-26ed7e6b65b6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:30.708963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ff861-81cd-41bf-882a-26ed7e6b65b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:30.726691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-201ff861-81cd-41bf-882a-26ed7e6b65b6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:30.727884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-201ff861-81cd-41bf-882a-26ed7e6b65b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:30.736903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 689/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:30.892922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1182d-f898-4752-809a-5a15baa0d630 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:30.907294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a1182d-f898-4752-809a-5a15baa0d630 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:30.912930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:30.919290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:30.919290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:30.919290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:30.923705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:30.925325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e447a-abe8-47d9-a599-edcc561218cc service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:30.938330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 730/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:21:31.889907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a410-ab39-4955-a6a8-ffe59f3f71b9 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:31.890682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a410-ab39-4955-a6a8-ffe59f3f71b9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:31.890682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a410-ab39-4955-a6a8-ffe59f3f71b9 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:31.891931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 690/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:31.897241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feef1b-b8f5-4ce6-b61e-041ae3e8e544 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:31.897736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feef1b-b8f5-4ce6-b61e-041ae3e8e544 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:31.897837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feef1b-b8f5-4ce6-b61e-041ae3e8e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:31.917763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-55feef1b-b8f5-4ce6-b61e-041ae3e8e544 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:31.919014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-55feef1b-b8f5-4ce6-b61e-041ae3e8e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:31.927094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 731/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:32.082512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a410-ab39-4955-a6a8-ffe59f3f71b9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:32.097100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a410-ab39-4955-a6a8-ffe59f3f71b9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:32.102719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:32.103642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:32.103765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:32.104012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:32.113979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:32.115855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7938b42-7d93-4cd4-86b8-e598b0632acf service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:32.127310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 691/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:33.087011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e71437-bbfa-4675-8f1c-21fd20cf278a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:33.087551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e71437-bbfa-4675-8f1c-21fd20cf278a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:33.087551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e71437-bbfa-4675-8f1c-21fd20cf278a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:33.089175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 732/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:33.094482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9b9e3-1e0f-411e-b37c-86f8db970bf9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:33.094935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9b9e3-1e0f-411e-b37c-86f8db970bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:33.095033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9b9e3-1e0f-411e-b37c-86f8db970bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:33.115042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7dd9b9e3-1e0f-411e-b37c-86f8db970bf9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:33.116241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7dd9b9e3-1e0f-411e-b37c-86f8db970bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:33.126285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 692/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:33.282711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e71437-bbfa-4675-8f1c-21fd20cf278a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:33.298667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e71437-bbfa-4675-8f1c-21fd20cf278a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:33.304182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:33.304826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:33.305113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:33.305419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:33.314354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:33.315956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fe41a-2c5c-40b1-a0e9-23071de1b2a5 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:33.326489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 733/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:34.250978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374662cf-0c57-4180-9729-86ed05e05396 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:34.251951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374662cf-0c57-4180-9729-86ed05e05396 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:34.251951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374662cf-0c57-4180-9729-86ed05e05396 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:34.252899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 693/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:21:34.258548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9aaa2-e74b-4b65-b726-03eca437a69a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:34.259358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9aaa2-e74b-4b65-b726-03eca437a69a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:34.259358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9aaa2-e74b-4b65-b726-03eca437a69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:34.290012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-eac9aaa2-e74b-4b65-b726-03eca437a69a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:34.292106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-eac9aaa2-e74b-4b65-b726-03eca437a69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:34.300092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 734/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:34.448702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374662cf-0c57-4180-9729-86ed05e05396 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:34.464087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374662cf-0c57-4180-9729-86ed05e05396 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:34.469591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:34.470237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:34.470527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:34.470816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:34.479784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:34.481408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423f224-91af-41f6-9197-7c204f8c1afd service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:34.491312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 694/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:21:35.451046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941be9d-bc3a-4041-a25a-be35efac9ad1 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:35.451638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941be9d-bc3a-4041-a25a-be35efac9ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:35.451638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941be9d-bc3a-4041-a25a-be35efac9ad1 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:35.453001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 735/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:35.458016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37319-6c9a-4bfe-ac85-a0acf9354eee None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:35.458396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37319-6c9a-4bfe-ac85-a0acf9354eee None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:35.458575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37319-6c9a-4bfe-ac85-a0acf9354eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:35.477086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dfc37319-6c9a-4bfe-ac85-a0acf9354eee None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:35.478464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dfc37319-6c9a-4bfe-ac85-a0acf9354eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:35.486549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 695/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:35.639872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941be9d-bc3a-4041-a25a-be35efac9ad1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:35.654253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941be9d-bc3a-4041-a25a-be35efac9ad1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:35.659623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:35.660259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:35.660541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:35.660814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:35.669489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:35.671077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba099f0f-0eef-4d57-8dae-2c0391414df3 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:35.680498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 736/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:36.616849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3919-f1fe-4247-8cb3-9a6d1484f255 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:36.617417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3919-f1fe-4247-8cb3-9a6d1484f255 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:36.617417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da3919-f1fe-4247-8cb3-9a6d1484f255 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:36.619232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 696/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:36.625678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623057d-0b52-4888-9254-0f863dac6735 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:36.626219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623057d-0b52-4888-9254-0f863dac6735 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:36.626219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623057d-0b52-4888-9254-0f863dac6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:36.645077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1623057d-0b52-4888-9254-0f863dac6735 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:36.646317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1623057d-0b52-4888-9254-0f863dac6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:36.654859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 737/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:36.808204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da3919-f1fe-4247-8cb3-9a6d1484f255 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:36.822543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da3919-f1fe-4247-8cb3-9a6d1484f255 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:36.827680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:36.828346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:36.828630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:36.828908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:36.837925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:36.839533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f70e67d-ed02-4719-8180-0e15b131f047 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:36.849618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 697/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:21:37.802699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7faa-cbf3-4578-89cc-0702977f4f7a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:37.803276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7faa-cbf3-4578-89cc-0702977f4f7a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:37.803276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af7faa-cbf3-4578-89cc-0702977f4f7a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:37.804606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 738/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:37.811344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a8b88-22d7-48df-9a6f-e38a7983e729 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:37.812069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a8b88-22d7-48df-9a6f-e38a7983e729 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:37.812473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a8b88-22d7-48df-9a6f-e38a7983e729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:37.834497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-873a8b88-22d7-48df-9a6f-e38a7983e729 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:37.835730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-873a8b88-22d7-48df-9a6f-e38a7983e729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:37.844626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 698/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:37.999963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af7faa-cbf3-4578-89cc-0702977f4f7a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:38.017065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af7faa-cbf3-4578-89cc-0702977f4f7a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:38.022972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:38.023524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:38.023810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:38.024089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:38.035046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:38.036702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e252a9-b50e-4b60-9172-78b0695607be service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:38.047729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 739/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:39.027146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46887f-6a2e-434d-9a2a-589a77fff0a7 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:39.027908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46887f-6a2e-434d-9a2a-589a77fff0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:39.027908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46887f-6a2e-434d-9a2a-589a77fff0a7 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:39.029057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 699/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:39.034082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c27c3-60bd-45e1-870d-365e97f2da14 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:39.034513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c27c3-60bd-45e1-870d-365e97f2da14 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:39.034667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c27c3-60bd-45e1-870d-365e97f2da14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:39.053133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2c1c27c3-60bd-45e1-870d-365e97f2da14 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:39.054360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2c1c27c3-60bd-45e1-870d-365e97f2da14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:39.062785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 740/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:39.219840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46887f-6a2e-434d-9a2a-589a77fff0a7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:39.235681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46887f-6a2e-434d-9a2a-589a77fff0a7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:39.241111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:39.241783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:39.242082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:39.242354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:39.251613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:39.253231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674704f8-2826-4fba-93a7-7cd3e4efe371 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:39.269061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 700/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:41.414114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bb5d5-eeed-4844-915f-34685fa1f116 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:41.414681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bb5d5-eeed-4844-915f-34685fa1f116 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:41.414681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bb5d5-eeed-4844-915f-34685fa1f116 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:41.416578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 741/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:21:41.422529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f8d3a-0679-42b7-aaf9-71dae0d216bf None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:41.423116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f8d3a-0679-42b7-aaf9-71dae0d216bf None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:41.423116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f8d3a-0679-42b7-aaf9-71dae0d216bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:41.443461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-278f8d3a-0679-42b7-aaf9-71dae0d216bf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:41.444669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-278f8d3a-0679-42b7-aaf9-71dae0d216bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:41.455627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 701/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:41.496830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bb5d5-eeed-4844-915f-34685fa1f116 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:41.507988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bb5d5-eeed-4844-915f-34685fa1f116 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:41.511667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:41.512303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:41.512590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:41.512864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:41.522204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:41.523710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526482-c037-4d64-bc89-226f21397494 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:41.534318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 742/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:43.931690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2b497-a8a0-476d-9801-498b5fae1df6 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:43.933240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2b497-a8a0-476d-9801-498b5fae1df6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:43.933240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2b497-a8a0-476d-9801-498b5fae1df6 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:43.935536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 702/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:43.941913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dc1a-50a5-4184-a9ef-64e0ec559443 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:43.942680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dc1a-50a5-4184-a9ef-64e0ec559443 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:43.942866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dc1a-50a5-4184-a9ef-64e0ec559443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:43.964966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b3b9dc1a-50a5-4184-a9ef-64e0ec559443 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:43.966249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b3b9dc1a-50a5-4184-a9ef-64e0ec559443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:43.978351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 743/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:44.022282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2b497-a8a0-476d-9801-498b5fae1df6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:44.033856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2b497-a8a0-476d-9801-498b5fae1df6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:44.037412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:44.038756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:44.038756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:44.038756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:44.048803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:44.050300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43502231-924b-4f57-b4d4-db91a4c54a47 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:44.061308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 703/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:46.761621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33243e60-95b5-423b-b656-942504118d84 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:46.762221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33243e60-95b5-423b-b656-942504118d84 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:46.762221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33243e60-95b5-423b-b656-942504118d84 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:46.763509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 744/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:46.768788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d8a7-206f-46f2-9801-3d6b23909856 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:46.769198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d8a7-206f-46f2-9801-3d6b23909856 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:46.769364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d8a7-206f-46f2-9801-3d6b23909856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:46.787618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0182d8a7-206f-46f2-9801-3d6b23909856 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:46.788867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0182d8a7-206f-46f2-9801-3d6b23909856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:46.797099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 704/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:46.835953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33243e60-95b5-423b-b656-942504118d84 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:46.846507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33243e60-95b5-423b-b656-942504118d84 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:46.849976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:46.850714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:46.850995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:46.851268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:46.860553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:46.862037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0047a9-c488-4aaf-9ff0-073d57c9c4df service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:46.872560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 745/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:49.236687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc960-b5fe-4e60-aa77-0840d82045b9 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:49.237220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc960-b5fe-4e60-aa77-0840d82045b9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:49.237220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbc960-b5fe-4e60-aa77-0840d82045b9 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:49.238536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 705/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:49.244299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713ad4d-1367-432d-ad67-f3c7d1b228c6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:49.244802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713ad4d-1367-432d-ad67-f3c7d1b228c6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:49.244912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713ad4d-1367-432d-ad67-f3c7d1b228c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:49.264695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e713ad4d-1367-432d-ad67-f3c7d1b228c6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:49.265931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e713ad4d-1367-432d-ad67-f3c7d1b228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:49.273673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 746/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:49.314257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc960-b5fe-4e60-aa77-0840d82045b9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:49.325160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbc960-b5fe-4e60-aa77-0840d82045b9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:49.328600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:49.329232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:49.329609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:49.329920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:49.340056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:49.341541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa766df9-9068-4a2b-be19-f2f96d8141b6 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:49.351782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 706/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:21:51.650588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd178e-4c56-4ebf-a132-438e193fb09f None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:51.651176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd178e-4c56-4ebf-a132-438e193fb09f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:51.651176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd178e-4c56-4ebf-a132-438e193fb09f None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:51.652524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 747/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:51.658628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc9ed-b742-4fa9-8f27-e33feb7d0163 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:51.659121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc9ed-b742-4fa9-8f27-e33feb7d0163 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:51.659239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dc9ed-b742-4fa9-8f27-e33feb7d0163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:51.677924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a59dc9ed-b742-4fa9-8f27-e33feb7d0163 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:51.679157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a59dc9ed-b742-4fa9-8f27-e33feb7d0163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:51.687898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 707/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:51.727715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd178e-4c56-4ebf-a132-438e193fb09f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:51.738915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd178e-4c56-4ebf-a132-438e193fb09f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:51.742558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:51.743902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:51.744192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:51.744559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:51.753800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:51.755292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920c5af-ee4e-40c7-a51f-6ea076ab9c0b service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:51.765646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 748/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:54.035632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cef56-b68d-4d13-9f03-84baa9feaede None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:54.036212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cef56-b68d-4d13-9f03-84baa9feaede None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:54.036212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cef56-b68d-4d13-9f03-84baa9feaede None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:54.037604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 708/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:54.043798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f695-73bb-4b2e-a9eb-9b15428642ef None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:54.044510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f695-73bb-4b2e-a9eb-9b15428642ef None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:54.044745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f695-73bb-4b2e-a9eb-9b15428642ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:54.064193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5ff2f695-73bb-4b2e-a9eb-9b15428642ef None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:54.065472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5ff2f695-73bb-4b2e-a9eb-9b15428642ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:54.074899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 749/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:54.114674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cef56-b68d-4d13-9f03-84baa9feaede None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:54.121201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=64 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:54.127274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cef56-b68d-4d13-9f03-84baa9feaede None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:54.131040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:54.131674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:54.131960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:54.132240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:54.141837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:54.143287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a370044-1edd-4588-bf22-62c41d154124 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:54.152932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 709/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:56.401751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b8fc-24f9-4b4b-9130-b5b2edf26d65 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:56.402315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b8fc-24f9-4b4b-9130-b5b2edf26d65 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:56.402315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b8fc-24f9-4b4b-9130-b5b2edf26d65 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:56.403707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 750/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:56.409194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aca7a-2645-4b37-8fa1-567f1f3e1ba2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:56.409663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aca7a-2645-4b37-8fa1-567f1f3e1ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:56.409793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aca7a-2645-4b37-8fa1-567f1f3e1ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:56.427400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6d6aca7a-2645-4b37-8fa1-567f1f3e1ba2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:56.428597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6d6aca7a-2645-4b37-8fa1-567f1f3e1ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:56.436476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 710/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:21:56.473889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3b8fc-24f9-4b4b-9130-b5b2edf26d65 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:56.479433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=73 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:21:56.485325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3b8fc-24f9-4b4b-9130-b5b2edf26d65 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:56.488798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:56.499336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:56.499336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:56.499336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:56.499646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:56.501014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a122dc-bc11-484f-9369-36aebe11d091 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:56.511024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 751/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:21:58.783334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402d6e-b54c-4fdf-ad65-b8354098451c None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:58.783826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402d6e-b54c-4fdf-ad65-b8354098451c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:58.783931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402d6e-b54c-4fdf-ad65-b8354098451c None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:58.785285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 711/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:21:58.791046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525889b-50dd-4d9f-b859-2d5afe250593 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:58.791678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525889b-50dd-4d9f-b859-2d5afe250593 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:58.791678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525889b-50dd-4d9f-b859-2d5afe250593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:58.810339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5525889b-50dd-4d9f-b859-2d5afe250593 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:21:58.811770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5525889b-50dd-4d9f-b859-2d5afe250593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:21:58.819550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 752/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:21:58.856704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402d6e-b54c-4fdf-ad65-b8354098451c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:21:58.868257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402d6e-b54c-4fdf-ad65-b8354098451c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:21:58.871772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:21:58.872426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:21:58.872707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:21:58.872980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:21:58.882546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:21:58.884824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee9698-db67-4ad1-8f4e-fcd05b0a891d service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:21:58.894957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 712/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:01.163211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a235ab-646f-4e84-a62a-9b2158c236d0 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:01.163773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a235ab-646f-4e84-a62a-9b2158c236d0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:01.164604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a235ab-646f-4e84-a62a-9b2158c236d0 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:01.166549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 753/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:01.172334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8d53-12fb-4a9d-aaa1-fb14d40363d7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:01.172749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8d53-12fb-4a9d-aaa1-fb14d40363d7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:01.172899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8d53-12fb-4a9d-aaa1-fb14d40363d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:01.190686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-fc6c8d53-12fb-4a9d-aaa1-fb14d40363d7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:01.191809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fc6c8d53-12fb-4a9d-aaa1-fb14d40363d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:01.199859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 713/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:01.237486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a235ab-646f-4e84-a62a-9b2158c236d0 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:01.248145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a235ab-646f-4e84-a62a-9b2158c236d0 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:01.252205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:01.252836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:01.253119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:01.253419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:01.262716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:01.264218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ac5d0-58c0-4097-90fa-1dbd1dbf755a service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:01.274123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 754/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:03.549809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc87d-a53c-4cf6-b207-c0e06dd00472 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:03.550381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc87d-a53c-4cf6-b207-c0e06dd00472 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:03.550381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfc87d-a53c-4cf6-b207-c0e06dd00472 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:03.551736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 714/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:03.557857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728218f8-f55c-4cf8-8a87-d688ce2e0b1d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:03.558342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728218f8-f55c-4cf8-8a87-d688ce2e0b1d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:03.558445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728218f8-f55c-4cf8-8a87-d688ce2e0b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:03.576808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-728218f8-f55c-4cf8-8a87-d688ce2e0b1d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:03.578066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-728218f8-f55c-4cf8-8a87-d688ce2e0b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:03.585971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 755/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:03.623316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc87d-a53c-4cf6-b207-c0e06dd00472 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:03.634217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfc87d-a53c-4cf6-b207-c0e06dd00472 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:03.637725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:03.638363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:03.638649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:03.638927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:03.648082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:03.649559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b624e5-6b89-4b82-aee0-1ce8abd51440 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:03.659262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 715/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:13.571596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=17 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:22:13.654929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=23 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:22:37.542068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b2d1-4e6b-4962-aeeb-5961f0c22841 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:37.542700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b2d1-4e6b-4962-aeeb-5961f0c22841 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:37.542854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b2d1-4e6b-4962-aeeb-5961f0c22841 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:37.545048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 756/1471] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:22:37.557052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9581-fb86-441b-98a8-903e4aaa38e1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:37.557718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9581-fb86-441b-98a8-903e4aaa38e1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:37.557908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb9581-fb86-441b-98a8-903e4aaa38e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:37.579718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-62eb9581-fb86-441b-98a8-903e4aaa38e1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:37.580974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-62eb9581-fb86-441b-98a8-903e4aaa38e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:37.593156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 716/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:37.641401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75452-9d77-4801-b55a-d9204aa3e061 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:37.641692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75452-9d77-4801-b55a-d9204aa3e061 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:37.641964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75452-9d77-4801-b55a-d9204aa3e061 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:37.643328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 757/1473] 10.222.0.42 () {54 vars in 962 bytes} [Fri May 27 16:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:37.649142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3805821-797d-4047-a2f8-49a695e89c28 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:37.649490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3805821-797d-4047-a2f8-49a695e89c28 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:37.649774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3805821-797d-4047-a2f8-49a695e89c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:37.669242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c3805821-797d-4047-a2f8-49a695e89c28 None None] MFA Rules not processed for user `e7017c7303f94b6380b99cb0b09d7e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:37.670486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c3805821-797d-4047-a2f8-49a695e89c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:37.677735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 717/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Fri May 27 16:22:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:37.685309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75452-9d77-4801-b55a-d9204aa3e061 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:37.697139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75452-9d77-4801-b55a-d9204aa3e061 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:37.700602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:37.701238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:37.701555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:37.701830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:37.711350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:37.712805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e140e-8101-42ec-ae74-20995b503056 service glance] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:37.722410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 758/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:22:40.075651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ad93-e0f5-45c8-b6fb-74e8c243bb64 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:40.076260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ad93-e0f5-45c8-b6fb-74e8c243bb64 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:40.076260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520ad93-e0f5-45c8-b6fb-74e8c243bb64 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:40.077721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 718/1476] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:22:40.084099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724357ab-6ca1-44ca-8363-a2edac5b161b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:40.084788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724357ab-6ca1-44ca-8363-a2edac5b161b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:40.084788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724357ab-6ca1-44ca-8363-a2edac5b161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:40.105630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-724357ab-6ca1-44ca-8363-a2edac5b161b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:40.106872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-724357ab-6ca1-44ca-8363-a2edac5b161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:40.115568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 759/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:40.142628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520ad93-e0f5-45c8-b6fb-74e8c243bb64 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:40.156991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520ad93-e0f5-45c8-b6fb-74e8c243bb64 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:40.162714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:40.163398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:40.163685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:40.163966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:40.164786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:40.166657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:22:40.166657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:22:40.166657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:22:40.167840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443cf31-cc19-4f09-8737-e006b619a24b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:40.188976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 719/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Fri May 27 16:22:40 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:22:42.456062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910cf-8d34-4347-ad24-bc433a71630a None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:42.456627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910cf-8d34-4347-ad24-bc433a71630a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:42.456627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb910cf-8d34-4347-ad24-bc433a71630a None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:42.458544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 760/1479] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:42.467279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d74f7-e098-4844-894a-fbb4b22a3641 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:42.467870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d74f7-e098-4844-894a-fbb4b22a3641 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:42.467870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d74f7-e098-4844-894a-fbb4b22a3641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:42.489094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-950d74f7-e098-4844-894a-fbb4b22a3641 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:42.490640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-950d74f7-e098-4844-894a-fbb4b22a3641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:42.498392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 720/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:42.543784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb910cf-8d34-4347-ad24-bc433a71630a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:42.558430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb910cf-8d34-4347-ad24-bc433a71630a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:42.563798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:42.564440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:42.564728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:42.565009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:42.574504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:42.576265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca96b1-2fc6-42ec-bb89-c25f4500153d service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:42.586812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 761/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:45.028465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b04d-7524-4409-9515-fb16c97ea526 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:45.029044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b04d-7524-4409-9515-fb16c97ea526 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:45.029044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0b04d-7524-4409-9515-fb16c97ea526 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:45.030430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 721/1482] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:45.035886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c93253-ee6e-40f3-b11b-2a15b271a5f7 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:45.036316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c93253-ee6e-40f3-b11b-2a15b271a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:45.036425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c93253-ee6e-40f3-b11b-2a15b271a5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:45.054115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-97c93253-ee6e-40f3-b11b-2a15b271a5f7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:45.055305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-97c93253-ee6e-40f3-b11b-2a15b271a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:45.063171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 762/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:45.099366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b04d-7524-4409-9515-fb16c97ea526 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:45.110251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0b04d-7524-4409-9515-fb16c97ea526 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:45.114088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:45.114870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:45.115160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:45.115441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:45.124305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:45.125784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93962b9-c614-4997-b6e7-e8aebe8258c8 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:45.136026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 722/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:47.418407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99ecc-73e9-4217-84d0-288b19e43c26 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:47.419038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99ecc-73e9-4217-84d0-288b19e43c26 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:47.419038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce99ecc-73e9-4217-84d0-288b19e43c26 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:47.420372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 763/1485] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:47.426491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d813d-1f41-41f9-be80-de2db6f41708 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:47.427007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d813d-1f41-41f9-be80-de2db6f41708 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:47.427123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d813d-1f41-41f9-be80-de2db6f41708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:47.447950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ff9d813d-1f41-41f9-be80-de2db6f41708 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:47.449313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ff9d813d-1f41-41f9-be80-de2db6f41708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:47.458822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 723/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:47.497260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce99ecc-73e9-4217-84d0-288b19e43c26 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:47.510698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce99ecc-73e9-4217-84d0-288b19e43c26 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:47.514365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:47.515015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:47.515307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:47.515588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:47.526362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:47.527990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5d439-f2ae-4f24-9f94-586ea484d5ca service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:47.538471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 764/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:49.924003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ee3a2-e3fe-40c8-bd81-8cc3019dc970 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:49.924541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ee3a2-e3fe-40c8-bd81-8cc3019dc970 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:49.924541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ee3a2-e3fe-40c8-bd81-8cc3019dc970 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:49.925910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 724/1488] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:22:49.931349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858011e9-7721-4bed-8b76-25a606540814 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:49.931838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858011e9-7721-4bed-8b76-25a606540814 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:49.932036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858011e9-7721-4bed-8b76-25a606540814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:49.950083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-858011e9-7721-4bed-8b76-25a606540814 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:49.956272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-858011e9-7721-4bed-8b76-25a606540814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:49.959826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 765/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:50.000311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ee3a2-e3fe-40c8-bd81-8cc3019dc970 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:50.011274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ee3a2-e3fe-40c8-bd81-8cc3019dc970 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:50.014688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:50.015314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:50.015597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:50.015872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:50.025234 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:50.026817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6290b-e09a-4712-8221-9dce8838606a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:50.036460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 725/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:22:52.654349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc914ec-2af3-4d72-aa53-9f4d0e113f14 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:52.654900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc914ec-2af3-4d72-aa53-9f4d0e113f14 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:52.654900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc914ec-2af3-4d72-aa53-9f4d0e113f14 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:52.656309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 766/1491] 10.222.0.42 () {54 vars in 926 bytes} [Fri May 27 16:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:22:52.662314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f309-6273-4e9a-9177-56c744926a5c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:52.662713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f309-6273-4e9a-9177-56c744926a5c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:52.662875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74f309-6273-4e9a-9177-56c744926a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:52.680328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9e74f309-6273-4e9a-9177-56c744926a5c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:22:52.681565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9e74f309-6273-4e9a-9177-56c744926a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:22:52.689295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 726/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Fri May 27 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:22:52.731108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc914ec-2af3-4d72-aa53-9f4d0e113f14 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:22:52.742666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc914ec-2af3-4d72-aa53-9f4d0e113f14 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:22:52.746156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:22:52.746796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:22:52.747083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:22:52.747364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:22:52.757413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:22:52.758854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d7fb3-e1e7-4cd1-bc4d-58651f137b29 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:22:52.768322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 767/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:23:02.675622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=33 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:23:02.764119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=33 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:24:50.299047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.299628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.299628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.317899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:24:50.319088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:50.332543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 727/1494] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:24:50.342266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.342800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.342800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.361226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:24:50.362444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:50.370445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 768/1495] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:24:50.375782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.385522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62abfc-8aea-4b22-a6df-739b852962ec None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.390215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.390859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.391140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.391415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.392151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:24:50.392453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.393616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.393616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:24:50.393616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:24:50.394519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.405940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 728/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:24:50.414102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.424315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da398-5ace-4cce-bfa2-0f644a504947 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.428939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.429635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.429921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.430196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.430929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.432726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.432726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:24:50.432726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:24:50.434101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.611649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 769/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:24:50 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:24:50.615927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.627245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecd841-02ea-43cc-9f9d-cef58697a1f2 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.685139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.685853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.686136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.686412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.687099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.688120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.688120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:24:50.688120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:24:50.689318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.781475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 729/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:24:50 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:24:50.785261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.796462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a602f9-54b2-48e9-980d-5b28a6451bd5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.800993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.801647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.801924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.802210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.802934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.803958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.803958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:24:50.803958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:24:50.804853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.820036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 770/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:24:50.824362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.834647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a5c97-47e9-459f-800e-9913acd00463 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.839080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.840221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.840576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.840957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.841706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.842711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.842711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:24:50.842711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:24:50.843589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e733542-210a-4c56-9149-767ae3645334 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.858621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 730/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:24:50.863011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:50.873144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8396-bf4a-464d-a617-c1f26a1e7339 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:50.878023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:50.878902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.879193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.879472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] PATH_INFO: `/v3/projects/33c64603c3c04ee3b10a6e5fa298e719/users/24e12aeb36dd4bcfae35438d2cab3376/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:50.894964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c64603c3c04ee3b10a6e5fa298e719, user_id=24e12aeb36dd4bcfae35438d2cab3376, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:50.896014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:50.896014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:24:50.896014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:24:50.898056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275dd61-8dcb-4e24-acb9-bffdb5644c59 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:50.970338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 771/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:24:50 2022] PUT /identity/v3/projects/33c64603c3c04ee3b10a6e5fa298e719/users/24e12aeb36dd4bcfae35438d2cab3376/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:24:50.983312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43238bb-4292-4e62-a2f5-e9e0500049ca None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:50.983606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43238bb-4292-4e62-a2f5-e9e0500049ca None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:50.990927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43238bb-4292-4e62-a2f5-e9e0500049ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.010599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e43238bb-4292-4e62-a2f5-e9e0500049ca None None] MFA Rules not processed for user `24e12aeb36dd4bcfae35438d2cab3376`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:24:51.018619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e43238bb-4292-4e62-a2f5-e9e0500049ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:51.077394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 731/1502] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:24:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:24:51.091062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffaa16-a208-4c88-94ff-05e8b0de97a6 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.091365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffaa16-a208-4c88-94ff-05e8b0de97a6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.093068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffaa16-a208-4c88-94ff-05e8b0de97a6 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.093068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 772/1503] 10.222.0.42 () {54 vars in 963 bytes} [Fri May 27 16:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:24:51.098204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d6f3-8cc7-417f-a32b-7de4a206d9c2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.098500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d6f3-8cc7-417f-a32b-7de4a206d9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.098789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2d6f3-8cc7-417f-a32b-7de4a206d9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.117193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-bdf2d6f3-8cc7-417f-a32b-7de4a206d9c2 None None] MFA Rules not processed for user `2be41ec2de3948e6a015dfe35c47a0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:24:51.118426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-bdf2d6f3-8cc7-417f-a32b-7de4a206d9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:51.162357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:24:51.170351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 732/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Fri May 27 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:24:51.283242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffaa16-a208-4c88-94ff-05e8b0de97a6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:51.294606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffaa16-a208-4c88-94ff-05e8b0de97a6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:51.298385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:51.299018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.299301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.299577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.307921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:51.310013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:51.318722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 773/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:24:51.361801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d779-6ab7-4a4f-8782-4c929021c332 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.362246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d779-6ab7-4a4f-8782-4c929021c332 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.362361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892d779-6ab7-4a4f-8782-4c929021c332 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.363678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 733/1506] 10.222.0.42 () {52 vars in 829 bytes} [Fri May 27 16:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:24:51.367495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:51.428943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be166e7b-c73e-4ab1-b8d2-505a0eb77f62 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:51.433664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:51.434532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.434817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.435093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] PATH_INFO: `/v3/users/24e12aeb36dd4bcfae35438d2cab3376` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.436288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e12aeb36dd4bcfae35438d2cab3376)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:51.437309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:51.437309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:24:51.437309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:24:51.438559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:51.627478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e12aeb36dd4bcfae35438d2cab3376'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:51.628134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e12aeb36dd4bcfae35438d2cab3376'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:51.683013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e12aeb36dd4bcfae35438d2cab3376'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:51.684426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e12aeb36dd4bcfae35438d2cab3376'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:51.703722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e12aeb36dd4bcfae35438d2cab3376'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:51.836851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:24:51.847409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 774/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:24:51 2022] DELETE /identity/v3/users/24e12aeb36dd4bcfae35438d2cab3376 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:24:51.851924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:51.852290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:51.852645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:51.874445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:24:51.875687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:51.936874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 734/1508] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:24:51.946353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:51.947816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:52.004438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab057083-bb74-422b-8cee-83b9d9394519 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:52.007969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:52.008626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:52.008913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:52.009193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:52.019086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:52.020560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90985d4-0e65-493c-9391-5d96b340d58f service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:52.031194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 775/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:24:52.519975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:24:52.523479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:24:52.534457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eddb64-8ee9-4f98-ac04-8e6d8d063f63 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:24:52.540340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:24:52.541214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:24:52.541542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:24:52.541856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] PATH_INFO: `/v3/projects/33c64603c3c04ee3b10a6e5fa298e719` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:24:52.543062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c64603c3c04ee3b10a6e5fa298e719)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:24:52.544100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:24:52.544100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:24:52.544100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:24:52.545356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:24:52.689921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c64603c3c04ee3b10a6e5fa298e719', 'request_id': 'req-4a39b787-7c90-4fb6-8232-987b98421d0f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:52.757175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c64603c3c04ee3b10a6e5fa298e719', 'request_id': 'req-4a39b787-7c90-4fb6-8232-987b98421d0f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:52.764767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c64603c3c04ee3b10a6e5fa298e719', 'request_id': 'req-4a39b787-7c90-4fb6-8232-987b98421d0f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:52.784003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] The token cache is being invalidate because project 33c64603c3c04ee3b10a6e5fa298e719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:24:52.784298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:52.785199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4a39b787-7c90-4fb6-8232-987b98421d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:24:52.786865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 735/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:24:52 2022] DELETE /identity/v3/projects/33c64603c3c04ee3b10a6e5fa298e719 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:25:02.026095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=29 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:25:02.780935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:26:35.666445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:35.667012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:35.667012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:35.668423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 776/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:35.673245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cac3a-13f7-4c18-b3be-51088b3d6639 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:35.679324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cac3a-13f7-4c18-b3be-51088b3d6639 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:35.679324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cac3a-13f7-4c18-b3be-51088b3d6639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:35.692471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8a0cac3a-13f7-4c18-b3be-51088b3d6639 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:35.693771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8a0cac3a-13f7-4c18-b3be-51088b3d6639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:35.754170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 736/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:35.930982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:35.932754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:36.010724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9eed-ccbe-4cbf-bb00-94afb194c850 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:36.016089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:36.016738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:36.017027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:36.017309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:36.026059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:36.027666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b088de-8fdc-4739-85c6-8f507aadcd22 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:36.037863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 777/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:36.842940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f110a1-446f-4362-ad72-685a9347ffd4 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:36.843474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f110a1-446f-4362-ad72-685a9347ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:36.843474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f110a1-446f-4362-ad72-685a9347ffd4 None None] PATH_INFO: `/v3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:36.844897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 737/1514] 10.222.0.15 () {54 vars in 925 bytes} [Fri May 27 16:26:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:26:36.851384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:36.851838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:36.851940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:36.870860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:36.872083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:36.926319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa611ab-a78e-4a5c-9791-663c327d9d5a None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:36.927988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa611ab-a78e-4a5c-9791-663c327d9d5a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:36.927988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa611ab-a78e-4a5c-9791-663c327d9d5a None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:36.929349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 738/1515] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:26:36.934438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522da2-8614-4eb7-9d9b-e00a8a7a831c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:36.934860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522da2-8614-4eb7-9d9b-e00a8a7a831c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:36.935213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0522da2-8614-4eb7-9d9b-e00a8a7a831c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:36.942476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 778/1516] 10.222.0.15 () {58 vars in 1015 bytes} [Fri May 27 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:36.958737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b0522da2-8614-4eb7-9d9b-e00a8a7a831c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:36.960003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b0522da2-8614-4eb7-9d9b-e00a8a7a831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:36.968404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 739/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:26:37.128175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:37.140978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523209f-6701-4cd6-b66c-e05d403c03c6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:37.147568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:37.148225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:37.148691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:37.148977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:37.157733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:37.159358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d648bbd-d51a-49ff-b34f-27e584a8d7c1 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:37.170005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 779/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:46.271425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0fbc1-282a-4649-920c-ffbcb0fe9171 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:46.271958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0fbc1-282a-4649-920c-ffbcb0fe9171 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:46.271958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0fbc1-282a-4649-920c-ffbcb0fe9171 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:46.273395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 740/1519] 10.222.0.42 () {54 vars in 960 bytes} [Fri May 27 16:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:46.278668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb659b-a547-43b1-b332-cced55e74595 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:46.279072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb659b-a547-43b1-b332-cced55e74595 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:46.279239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb659b-a547-43b1-b332-cced55e74595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:46.297670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-94bb659b-a547-43b1-b332-cced55e74595 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:46.298951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-94bb659b-a547-43b1-b332-cced55e74595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:46.307539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 780/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Fri May 27 16:26:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:46.315170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0fbc1-282a-4649-920c-ffbcb0fe9171 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:46.329216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0fbc1-282a-4649-920c-ffbcb0fe9171 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:46.335337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:46.335979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:46.336268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:46.336548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:46.345421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:46.347081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0be68b-88e0-4f0e-99f6-ff795ce74c25 service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:46.358152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 741/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:48.582907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837724e-8c74-4411-b70e-2ba13f2ec976 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:48.583483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837724e-8c74-4411-b70e-2ba13f2ec976 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:48.583735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837724e-8c74-4411-b70e-2ba13f2ec976 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:48.585116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 781/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:48.590786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159832-8383-434e-ad1a-597c7836ebd1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:48.591269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159832-8383-434e-ad1a-597c7836ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:48.591413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1159832-8383-434e-ad1a-597c7836ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:48.609819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c1159832-8383-434e-ad1a-597c7836ebd1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:48.611038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c1159832-8383-434e-ad1a-597c7836ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:48.619508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 742/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:26:48.773148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837724e-8c74-4411-b70e-2ba13f2ec976 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:48.788303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837724e-8c74-4411-b70e-2ba13f2ec976 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:48.793870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:48.794601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:48.794890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:48.795169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:48.804640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:48.807835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59475e-1875-478d-9b26-3ce558f2112d service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:48.816012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 782/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:49.691177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dde6e-8542-4010-9e07-b920b5a5bd68 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:49.691761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dde6e-8542-4010-9e07-b920b5a5bd68 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:49.691761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dde6e-8542-4010-9e07-b920b5a5bd68 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:49.693419 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 743/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 16:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:49.698141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30c5c-c746-4092-85f0-85fc6874cebf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:49.698568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30c5c-c746-4092-85f0-85fc6874cebf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:49.698702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b30c5c-c746-4092-85f0-85fc6874cebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:49.716696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-88b30c5c-c746-4092-85f0-85fc6874cebf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:49.717944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-88b30c5c-c746-4092-85f0-85fc6874cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:49.726194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 783/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 16:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:49.878523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dde6e-8542-4010-9e07-b920b5a5bd68 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:49.893453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dde6e-8542-4010-9e07-b920b5a5bd68 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:49.900030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:49.900681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:49.900970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:49.901260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:49.910011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:49.912520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c7476-ec39-422a-8908-c177e2e4fe86 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:49.922551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 744/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:54.114661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:54.115267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:54.115267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:54.116715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 784/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:54.122119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2706e0-bad1-46ce-b244-f14ed9292f0b None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:54.122582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2706e0-bad1-46ce-b244-f14ed9292f0b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:54.122724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2706e0-bad1-46ce-b244-f14ed9292f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:54.141056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6b2706e0-bad1-46ce-b244-f14ed9292f0b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:54.142390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b2706e0-bad1-46ce-b244-f14ed9292f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:54.150704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 745/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:54.186604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:54.188352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:54.236646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:26:54.249446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9081-98c2-4400-b7f1-1e612a7f121e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:54.252874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:54.253551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:54.253837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:54.254116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:54.263154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:54.264599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae472e-a947-4f57-958e-99901326ff23 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:54.274481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 785/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:26:56.492277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:56.492839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:56.492839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:56.494226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 746/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:26:56.499554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ceee-7015-4988-829d-ee66f7d84d39 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:56.499944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ceee-7015-4988-829d-ee66f7d84d39 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:56.500128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722ceee-7015-4988-829d-ee66f7d84d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:56.518955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c722ceee-7015-4988-829d-ee66f7d84d39 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:26:56.520171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c722ceee-7015-4988-829d-ee66f7d84d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:56.528856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 786/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:26:56.565152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:26:56.566895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:26:56.580450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aba45-ad39-4ddb-b4cf-a5023a1ee252 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:26:56.584626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:26:56.585262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:26:56.585574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:26:56.585853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:26:56.595190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:26:56.596626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4b38-5e8c-442f-9a43-e9b5865d29ca service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:26:56.606691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 747/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:27:00.455271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:00.455844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:00.455844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:00.457225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 787/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:27:00.463495 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed7844-0ad0-40a8-9da7-807ee7111319 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:00.463933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed7844-0ad0-40a8-9da7-807ee7111319 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:00.464067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed7844-0ad0-40a8-9da7-807ee7111319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:00.483379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-feed7844-0ad0-40a8-9da7-807ee7111319 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:27:00.484628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-feed7844-0ad0-40a8-9da7-807ee7111319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:27:00.492656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 748/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:27:00.772900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:27:00.774750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:27:00.831297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aebdd-c2e4-4a7f-b1ae-7af2dfa8a117 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:27:00.836391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:27:00.837156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:00.837474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:00.837756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:00.847076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:27:00.848510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7700e2-4f62-4ed8-8fc3-3fa3b5eee033 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:27:00.858871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 788/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:27:03.115203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:03.115793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:03.115793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:03.117141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 749/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 16:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:27:03.122459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d38859-5d04-4679-acab-ae6d1c142766 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:03.122871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d38859-5d04-4679-acab-ae6d1c142766 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:03.123030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d38859-5d04-4679-acab-ae6d1c142766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:03.142508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c1d38859-5d04-4679-acab-ae6d1c142766 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:27:03.143702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c1d38859-5d04-4679-acab-ae6d1c142766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:27:03.152273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 789/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:27:03.421637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:27:03.423359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:27:03.431756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=58 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:27:03.437560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b44a58-e721-449b-ac96-b636e5ebf5c8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:27:03.440978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:27:03.441639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:27:03.441920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:27:03.445278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:27:03.451841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:27:03.453261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9777c9e-7576-49f6-aa79-507b472f4d97 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:27:03.463181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 750/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:27:13.137682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=39 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:27:13.466996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:35.810829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:35.811930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:35.812430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:35.838018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:35.839240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:35.854808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 790/1540] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:35.866052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88debc-d448-49a0-919c-55fe3fdf80e0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:35.866634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88debc-d448-49a0-919c-55fe3fdf80e0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:35.866634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88debc-d448-49a0-919c-55fe3fdf80e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:35.891589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7d88debc-d448-49a0-919c-55fe3fdf80e0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:35.893194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7d88debc-d448-49a0-919c-55fe3fdf80e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:35.904715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 751/1541] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:35.909442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:35.921395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20193c61-cced-4fc6-86f9-c2bc88491e28 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:35.937254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:35.937983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:35.938269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:35.938551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:35.939313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:35.939625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:35.940684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 16:28:35.940684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:28:35.940684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:28:35.952730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:35.969436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 791/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:35.987995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ac5d-4167-40f0-bb47-2d6b4c93d015 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:35.988917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ac5d-4167-40f0-bb47-2d6b4c93d015 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:35.989454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ac5d-4167-40f0-bb47-2d6b4c93d015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:35.989581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.000462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b2133-6c48-4384-bd52-3233669df566 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.006099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.006799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.007086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.007365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.008075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.013368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.022954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7c38ac5d-4167-40f0-bb47-2d6b4c93d015 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.024587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7c38ac5d-4167-40f0-bb47-2d6b4c93d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.049427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 752/1543] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.060753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.061436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.061936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.089076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.090703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.105520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 753/1544] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.112679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.118271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 792/1545] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:35 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:28:36.122683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.124430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64ceb7-8fc0-4e11-8327-8ef6b15dca88 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.228392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.229405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.229887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.230356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.231272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.231783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.233758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bad7b-2f95-49cd-9aa9-87a446b9b29b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.253006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 754/1546] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:28:36.255396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.255887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.256461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.268005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fdaf3-1263-4e43-82af-61dc555221c5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.282382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.283934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.284143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.284974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.285329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.285329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.286644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.291544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:28:36.291544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:28:36.291544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:28:36.291544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60f0a9-b649-4276-b9e6-bdd4bfc3e345 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.300655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 755/1547] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.302379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.314494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae09197-6514-4eb4-902a-8ec26578d1c3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.320163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.320833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.321120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.321531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.322228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.323370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:28:36.323370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:28:36.323370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:28:36.324736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337f268-8797-42ef-b8bb-c89dea1459f1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.366403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 793/1548] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:28:36.367689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205461-6e8e-4f05-ab1e-84671b4c0ab6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.367979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205461-6e8e-4f05-ab1e-84671b4c0ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.368257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be205461-6e8e-4f05-ab1e-84671b4c0ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.390864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-be205461-6e8e-4f05-ab1e-84671b4c0ab6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.392112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-be205461-6e8e-4f05-ab1e-84671b4c0ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.401567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 794/1549] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.402870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b357b4c-69e1-4105-a299-d4854718f016 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.403171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b357b4c-69e1-4105-a299-d4854718f016 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.403452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b357b4c-69e1-4105-a299-d4854718f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.414080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 756/1550] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:36.415692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1244f8d-255f-47f3-87af-0477cdb02fdf None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.415996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1244f8d-255f-47f3-87af-0477cdb02fdf None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.416279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1244f8d-255f-47f3-87af-0477cdb02fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.425949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6b357b4c-69e1-4105-a299-d4854718f016 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.427256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6b357b4c-69e1-4105-a299-d4854718f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.434945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b1244f8d-255f-47f3-87af-0477cdb02fdf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.436083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b1244f8d-255f-47f3-87af-0477cdb02fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.486981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 795/1551] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.490741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.491055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.491331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.498804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 757/1552] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.500195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd433-fa74-4845-b785-25356eb5c126 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.500489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd433-fa74-4845-b785-25356eb5c126 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.500766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebd433-fa74-4845-b785-25356eb5c126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.512485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.513716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.519594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c5ebd433-fa74-4845-b785-25356eb5c126 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.521647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c5ebd433-fa74-4845-b785-25356eb5c126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.522357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 796/1553] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.522966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.533843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 758/1554] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.534695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3169-9213-4034-b5e8-5a5e0e8f5db9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.535204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.535582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.536056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.539845 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.540507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.540839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.541057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.541808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.544331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025e559-ba1a-41b4-85e0-551563476f10 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.553635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 797/1555] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:36.554620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.554869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b3570-dd70-4081-850a-d49bdd9b0fcd None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.555156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b3570-dd70-4081-850a-d49bdd9b0fcd None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.555429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b3570-dd70-4081-850a-d49bdd9b0fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.555828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.564672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 759/1556] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.565362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.574859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-671b3570-dd70-4081-850a-d49bdd9b0fcd None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.576245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-671b3570-dd70-4081-850a-d49bdd9b0fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.578065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9e2c6-8890-4a3d-8316-5e9021c614ef None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.583215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.583878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.584166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.584437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.587611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 798/1557] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.587754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.587846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.587846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.587846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.588500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.604933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.606150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.615342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 799/1558] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.615342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.623772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7fd63-bd70-4bfb-ad65-7b99de32da5f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.628206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.628826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.629110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.629956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.630661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.630971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.632556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508bc7-7084-4ebd-9ed7-2b0b0ad9b4fb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.639243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:36.644098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 800/1559] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:36.646131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebf41-42be-48ad-bf06-92fd04a704f6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.646419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebf41-42be-48ad-bf06-92fd04a704f6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.646699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebf41-42be-48ad-bf06-92fd04a704f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.664254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ea4ebf41-42be-48ad-bf06-92fd04a704f6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.665475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ea4ebf41-42be-48ad-bf06-92fd04a704f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.673158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 801/1560] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.674418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.679701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.679701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.692828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.693919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.702353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 802/1561] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.708689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.712946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986459b-f3b6-415e-a3b4-7267400310d5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.717568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.718201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.718483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.718756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.719472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.721051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb08911-dea4-4e9f-a379-31a87c3591c1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.731044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 803/1562] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:36.737728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.737728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.737728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.738083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 760/1563] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:36.738083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.748113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd2ee-c469-4dd0-a23a-e14b1fd8ef66 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.752472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:36.752809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.753507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.753708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:36.753871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.754151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.755408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.755720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.757301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.765331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 804/1564] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:36.766820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.771705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 761/1565] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:36.771705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.776036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:36.777559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0576b24-dac2-4168-94c3-ced2de2ae848 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.782424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3e57d-a6e4-4b10-b065-97c1dd2f073a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.782577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.783170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.783461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.783741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.784469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.784781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.787015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.787223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.787745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.788031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.788305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.788990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.790945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.799635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 805/1566] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:36.802771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.812751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb209-9e15-4215-bdff-c2d4fe985e8e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.817886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.818606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.818917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.819199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.819906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.820214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.821886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.835035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 806/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:36.835892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.848878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbed99a-209b-4778-9ff2-8e5a41ea1f3e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.854816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.855729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.856205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.856799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.857861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:36.864266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.864266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.874569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 807/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:36.875480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.887940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a323071-d9ae-4646-b139-b546c26fbf12 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.889210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 762/1569] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:36 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:36.890071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.963693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da036640-646b-4b2e-91ac-5ae5ebcf1916 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.964652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.965330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.965634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.965984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.966789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.968404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:36.969097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:36.970019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:36.970305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:36.970585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] PATH_INFO: `/v3/projects/26ab6251a87c4271b77242545ef4837f/users/391d41dab9294cd39d3b80d60570034e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:36.981715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 808/1570] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:36.982661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:36.990954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ab6251a87c4271b77242545ef4837f, user_id=391d41dab9294cd39d3b80d60570034e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:36.992584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:28:36.992584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:28:36.992584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:28:36.995239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955bca-6ee5-4437-a23a-cdde88c56e70 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:36.995371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.000165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.000823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.001106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.001414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.002137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:28:37.002444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.004043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.018291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 809/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:28:37.019110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.029174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56748d0c-51d0-4c77-9eb3-26b80e013557 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.035659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.035659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.035659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.035659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.036553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.038376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.052767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 763/1572] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:36 2022] PUT /identity/v3/projects/26ab6251a87c4271b77242545ef4837f/users/391d41dab9294cd39d3b80d60570034e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:37.053858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.117084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 810/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.121646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.124200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e31e7-81c9-4ecd-b53b-02706e581432 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.173802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:37.185120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.190407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.190407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.190407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.190407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.190788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badb0cc-509d-4230-8cb6-dcaa4b1f8260 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.190879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.192907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.193681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.193970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.194248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.195972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.197811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.204584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:37.282333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 764/1574] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.283172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.296496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 811/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.296496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.354917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11451b7-38b5-4d35-a3d1-1373cf5bedf5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.355459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade57b-d548-4ad9-9e55-2aae5f15ee69 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.360017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.360422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.361094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.361268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.361415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.361687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.361794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.362039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.362383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.362728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.364927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.365057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e99ad8-cc6d-49a5-bbcf-f249d3783969 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.469429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 812/1576] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.470471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.481667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8953d3b-2db7-4d11-bbc3-f6652007ea8f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.486151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.486795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.487075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.487348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.488057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.490531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.500292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 813/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:37.501184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.511795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7700ab-3160-4138-9aa8-b8d71e6abe69 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.515203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 765/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/projects => generated 391 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.515203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.515203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.515203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.531704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:37.535413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:37.573711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.574376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.574664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.574943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.575622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.577572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.596422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 766/1579] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:37.597245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.607109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6439ff-87e7-43dd-b10c-3dbd81d3678e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.611864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.612532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.612821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.613104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.614382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.616460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.653253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 814/1580] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.654102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.695059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:37.709441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 767/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.710517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.720074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eac1-8f3d-4142-b126-c1fac3c81663 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.720798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9424c-def9-4277-95a2-bfc963be7c71 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.726277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.726875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.727004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.727234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.727519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.727682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.727798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.728082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.728194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.728770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.730256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.730971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.863196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 768/1582] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.863467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 815/1583] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:37.864401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.865168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.875723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899bda1-43eb-4880-a4c6-9677be505f87 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.880261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55819e8-2df6-43d4-b98c-c4203fee929f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.881522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.881713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.882187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.882472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.882674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.882788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.882967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.883244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] PATH_INFO: `/v3/projects/5dd9243059c8473c873d19c3a18d3f85/users/ef7185d37eef4e48a716bbe8a8c39125/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.883468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.885108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.897717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 769/1584] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:37.898584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:37.901271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd9243059c8473c873d19c3a18d3f85, user_id=ef7185d37eef4e48a716bbe8a8c39125, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.904015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.908733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0652a-c2eb-4847-bf6a-fd1dc3d15a2a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:37.913592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:37.914264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:37.914550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:37.914831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:37.915584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:37.917538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:37.995211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 816/1585] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:37 2022] PUT /identity/v3/projects/5dd9243059c8473c873d19c3a18d3f85/users/ef7185d37eef4e48a716bbe8a8c39125/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:37.996010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.012942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 770/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:37 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:38.014306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.026914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eace4-37a6-48ab-aa24-1384b29acac5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.040083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:38.055568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7d78-b08e-46dd-bde7-e8254be98172 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.060558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.061259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.061631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.061985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.086384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.099075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.099075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.099075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.099075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.099075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.125130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.126579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.137281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 817/1587] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:38.138103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.149118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca67b81-8125-44bb-81bd-50d7f8d3a272 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.155000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.158231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.158231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.158231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.158231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.158706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.169145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 818/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.170168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.179901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce3c7b-92e4-4dea-af5e-d842cf17a8dc admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.184584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.185251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.186992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.186992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.186992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.188138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.189755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 771/1589] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:38 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:38.190590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.201803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732410fd-b4a0-4667-b5c2-5c4eaa477c58 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.204055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 819/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.204055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.209319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.210330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.210620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.210897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.211622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.213229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f2bd2-3b12-409c-9cf5-121689b6b827 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.215602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46e370-937f-4733-9f17-fc555290783e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.219197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.219900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.220190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.220469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.221202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.222998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.224426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 772/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.226148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.226436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.226716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.236328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 820/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.237167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.248377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbcbda-9fa9-41a5-a4ee-9dcd7a36af45 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.253480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.253663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] MFA Rules not processed for user `ef7185d37eef4e48a716bbe8a8c39125`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:38.254152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.254447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.255278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.255491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:38.256182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.257838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.269316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 821/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.269971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.282114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50eed5-e638-49f8-bf5c-6fb7d57d72af admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.288881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.289942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.290397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.290738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.291515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.293388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.295947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:38.306149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 822/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.306149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.318116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10294b80-01d8-4a22-a70e-6b31e6ae6b55 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.325381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.326599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.327072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.327737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.328992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.331721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.344418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 823/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.346807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 773/1596] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:38.348291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.351258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.363412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb33944-7a49-4fe5-8c2e-b198e57275ca admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.366254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71963406-8839-4374-99a0-4c74f71d83a7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.367278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.368015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.368309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.368631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.369394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.371098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.374820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.375648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.375910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.376177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.387729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 824/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.387941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.388096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.389641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.400592 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cee20-e938-499a-a96f-8c4a40461830 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.407348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.408246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.408551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.408807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] PATH_INFO: `/v3/projects/b9f649e42c7d41758c6c06a73435a96d/users/0c9fa62de02d4149aa0ceb4636628ea4/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.410105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 774/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.411645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.423188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b581b6-b45d-4500-b7a1-c9dd1ad58cd3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.426576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f649e42c7d41758c6c06a73435a96d, user_id=0c9fa62de02d4149aa0ceb4636628ea4, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.427793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.428494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.428781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.429060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.429401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.429813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.431415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.444966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 775/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.444966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.454727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27493742-f301-4cd4-8828-7edab1b7cf64 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.460487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.461402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.461689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.461968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] PATH_INFO: `/v3/projects/32666b3567c94e06a7f720c1abe2a8b7/users/a0a422ada9ab43609a7940a1df349b7b/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.477863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=32666b3567c94e06a7f720c1abe2a8b7, user_id=a0a422ada9ab43609a7940a1df349b7b, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.480627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6caa6a1-32dd-4bdd-a8c5-46a16cef57cb admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.517289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 825/1600] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:38 2022] PUT /identity/v3/projects/b9f649e42c7d41758c6c06a73435a96d/users/0c9fa62de02d4149aa0ceb4636628ea4/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:38.518331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.588747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354611bd-295a-4162-a517-6c5365b5eb15 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.596330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 776/1601] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:38 2022] PUT /identity/v3/projects/32666b3567c94e06a7f720c1abe2a8b7/users/a0a422ada9ab43609a7940a1df349b7b/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:38.598212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfec4a-a248-4383-94ba-340efa6f5fa0 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.599786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfec4a-a248-4383-94ba-340efa6f5fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.600246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfec4a-a248-4383-94ba-340efa6f5fa0 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.603124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 777/1602] 10.222.0.42 () {54 vars in 961 bytes} [Fri May 27 16:28:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:38.604202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfec4a-a248-4383-94ba-340efa6f5fa0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.657020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:38.661539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.661947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.662230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.662501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.663216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.664914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.675636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 826/1603] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:38.675811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.684348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfec4a-a248-4383-94ba-340efa6f5fa0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.686879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5ef89-6f23-4bb8-9d7e-daf5cdd75dbb admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.690550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.691482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.691764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.692561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] PATH_INFO: `/v3/projects/c409475791e243108336a3cd2e25d3ac/users/56799d191ffe4193918fa0221c27c0e0/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.693745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.694647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.695031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.695316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] PATH_INFO: `/v3/projects/3fa953c1f6d240098f164785b8ccc5b4/users/87d41932bfa94f369103ed29d29940b3/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.710117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c409475791e243108336a3cd2e25d3ac, user_id=56799d191ffe4193918fa0221c27c0e0, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.712977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186076ed-4624-434f-aedd-0945343e1e86 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.713733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa953c1f6d240098f164785b8ccc5b4, user_id=87d41932bfa94f369103ed29d29940b3, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.716724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.853797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 827/1604] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:38 2022] PUT /identity/v3/projects/3fa953c1f6d240098f164785b8ccc5b4/users/87d41932bfa94f369103ed29d29940b3/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:38.854197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 778/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:38 2022] PUT /identity/v3/projects/c409475791e243108336a3cd2e25d3ac/users/56799d191ffe4193918fa0221c27c0e0/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:38.858514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:38.858628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230cb6-f4f5-4177-9b37-407efb437ca1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.858628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230cb6-f4f5-4177-9b37-407efb437ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.858628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230cb6-f4f5-4177-9b37-407efb437ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.884441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-37230cb6-f4f5-4177-9b37-407efb437ca1 None None] MFA Rules not processed for user `0c9fa62de02d4149aa0ceb4636628ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:38.892950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-37230cb6-f4f5-4177-9b37-407efb437ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:38.893104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:38.922293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707869ad-7ef1-45ca-ac30-3dd21bb92a0c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:38.927786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:38.928709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.929091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.929393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] PATH_INFO: `/v3/projects/6b6ae55c00cb4a43b8e2bc7df2ea9626/users/cf3e0adc32814585b57bf9f884840fc9/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.949393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b6ae55c00cb4a43b8e2bc7df2ea9626, user_id=cf3e0adc32814585b57bf9f884840fc9, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:38.952224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:38.970377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 779/1606] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:38.971693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c00f7d-cde1-4ee3-8e53-b052acbb54b5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:38.972002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c00f7d-cde1-4ee3-8e53-b052acbb54b5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:38.972275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c00f7d-cde1-4ee3-8e53-b052acbb54b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:38.995356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c2c00f7d-cde1-4ee3-8e53-b052acbb54b5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:38.996598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c2c00f7d-cde1-4ee3-8e53-b052acbb54b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.004447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 780/1607] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.007095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa715b1-d026-46ac-900f-d32bf183bb0f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.007402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa715b1-d026-46ac-900f-d32bf183bb0f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.007787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa715b1-d026-46ac-900f-d32bf183bb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.028563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4aa715b1-d026-46ac-900f-d32bf183bb0f None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:39.029796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4aa715b1-d026-46ac-900f-d32bf183bb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.064872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 828/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:38 2022] PUT /identity/v3/projects/6b6ae55c00cb4a43b8e2bc7df2ea9626/users/cf3e0adc32814585b57bf9f884840fc9/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:39.066591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.104302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 781/1609] 10.222.0.42 () {58 vars in 1058 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.105719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9ef8-094a-425a-af63-20458f0de75c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.105996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9ef8-094a-425a-af63-20458f0de75c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.106275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9ef8-094a-425a-af63-20458f0de75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.124291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-32bc9ef8-094a-425a-af63-20458f0de75c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:39.125553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-32bc9ef8-094a-425a-af63-20458f0de75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.136781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac289ff7-ff85-456c-a604-17935765aed3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.141772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.142672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.142953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.143233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] PATH_INFO: `/v3/projects/5931bc4fc35642bdafa6391e2f99138c/users/3bd0a732c554439eb1434edc17e1b6dc/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.159524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:39.159763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5931bc4fc35642bdafa6391e2f99138c, user_id=3bd0a732c554439eb1434edc17e1b6dc, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.163418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.187150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 782/1610] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.191533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.191827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.192448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.210204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:39.211434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.222817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 783/1611] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.224400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.234240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d2469-dcc3-40d7-8abc-7a369c10c47c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.238812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.239509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.239825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.240065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.240753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.242965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24269d57-c6b9-4494-baeb-f9583b21223b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.247089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 829/1612] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:39 2022] PUT /identity/v3/projects/5931bc4fc35642bdafa6391e2f99138c/users/3bd0a732c554439eb1434edc17e1b6dc/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:39.249507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.316635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621325da-9f3c-4e57-957b-4bc8be7d00f0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.320928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.321643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.321930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.322229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.359843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 784/1613] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:39.361145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.361477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.361766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.373872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:39.383240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:39.384422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.387941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.389674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.401497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 830/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:39.402237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.453069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 785/1615] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.453069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.482272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05c65-fe9a-4e56-9918-9f6c569c1a74 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.510275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c500b-23bf-47cc-876e-5f5b5c1d8018 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.514197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.514910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.515121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.515393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.525581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.527022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.537072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 786/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:39.537970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.546140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8c866-845c-42a0-a85f-187f72fa310f service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.550832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.551424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.552080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.552718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.566908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.568239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.568524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.568801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.579472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.580966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401db936-1db5-4572-a52c-9c5e26ffc108 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.593639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 787/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:39.595202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.595331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.595576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.630261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.632153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.634494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] MFA Rules not processed for user `3bd0a732c554439eb1434edc17e1b6dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:39.636085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:39.644549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 831/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:39.645806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.682058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09b7a-ece9-4cec-b175-1fa47deceb7a service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.688708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.690429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.690794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.691139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.691962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.694384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.717801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:39.759007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 788/1619] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:39.760033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.772993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2184ed8-77b3-4c00-92c5-2502ceeb6fe5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.777542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.778182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.779438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.779438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.789651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.791374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.813777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 789/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:39.814648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.836636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877da4b9-6ebd-49db-8a24-50d2cf9d1667 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.852429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.853439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.853900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.854345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.855242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.860728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 832/1621] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:39.861507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c382c-0603-4a8c-bdf2-41d43eaf731a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.861826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.876805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:39.886018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97c76a-deed-470e-9f43-92cad6a10651 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.900325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.901009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.901302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.901732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.902475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.904173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea73a0f-2961-431a-80c0-e596cfbde81f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:39.931921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 833/1622] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:39.935262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934624-c7b9-4b0a-8d8a-b0b59010094d None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.935556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934624-c7b9-4b0a-8d8a-b0b59010094d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.936217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934624-c7b9-4b0a-8d8a-b0b59010094d None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.937371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 834/1623] 10.222.0.42 () {54 vars in 912 bytes} [Fri May 27 16:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:39.945758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934624-c7b9-4b0a-8d8a-b0b59010094d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:39.960093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934624-c7b9-4b0a-8d8a-b0b59010094d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:39.967155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:39.969984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:39.970273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:39.970710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] PATH_INFO: `/v3/projects/72a58ff568234bc5b242e68922945b3c/users/38f057c4125a4f4ab14d506ead87fa1f/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:39.988257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a58ff568234bc5b242e68922945b3c, user_id=38f057c4125a4f4ab14d506ead87fa1f, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:39.993307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.012463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 790/1624] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:39 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:40.013874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.014181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.014466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.045858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:40.047107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:40.100701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 835/1625] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:39 2022] PUT /identity/v3/projects/72a58ff568234bc5b242e68922945b3c/users/38f057c4125a4f4ab14d506ead87fa1f/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:40.101906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.123579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 791/1626] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:40.124469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.208445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cddaa9-672b-4f76-81f1-a2a1e705b22a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.225112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.227357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.227357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.229099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.230184 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.232305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.237144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91415bd-f6a9-44ae-8caf-1f15167a4032 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.249581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.250289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.251083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.251376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.252719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.254315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.272245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 792/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:40.273015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.302633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:40.345613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf02a77-8eae-47c5-b737-d3eeea3bd48d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.351496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.352521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.352521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.352766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.388013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 836/1628] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:40 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:40.388013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.415722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a69edf-2c2e-4307-9f53-c77207cb5b5e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.434142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.434867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.435151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.435516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] PATH_INFO: `/v3/projects/72a58ff568234bc5b242e68922945b3c/users/38f057c4125a4f4ab14d506ead87fa1f/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.437667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a58ff568234bc5b242e68922945b3c, user_id=38f057c4125a4f4ab14d506ead87fa1f, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.440519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.457399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.459321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0f61-5e57-45e9-b1b8-cf7d78a3d807 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.460941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 837/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:40 2022] PUT /identity/v3/projects/72a58ff568234bc5b242e68922945b3c/users/38f057c4125a4f4ab14d506ead87fa1f/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:40.462250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.491098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 793/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:40.495814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.496461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.497417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.590537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] MFA Rules not processed for user `38f057c4125a4f4ab14d506ead87fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:40.592449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:40.684054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24d2f-0e13-42ee-bf5f-c801a3c41266 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.696732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.696732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.696732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.696732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.697133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.699412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.744201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 838/1631] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:40.747920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.774767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb876dee-ee84-4a83-a280-e7f2b201097b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.798158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.799572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.799826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.800329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:40.802286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:40.812781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:40.846450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 839/1632] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:40.851777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 794/1633] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:40.853559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.875695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:40.920030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2dd7f-e151-4eaa-834e-f29718c3b215 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:40.960619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:40.962249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:40.962841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:40.963264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] PATH_INFO: `/v3/projects/bf040b2afd06492589c6028ef79a594d/users/6afb908a50d54be184a5b1e1f0430e6c/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.020290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf040b2afd06492589c6028ef79a594d, user_id=6afb908a50d54be184a5b1e1f0430e6c, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:41.026168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:41.032642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ae67e-96d5-43e2-b555-70f1377430c3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:41.039235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:41.040250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:41.040610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:41.040963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.053118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:41.054909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e9f1c-1676-4249-90b9-87cf430db65d service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:41.077391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 795/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:28:41.140314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 840/1635] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:40 2022] PUT /identity/v3/projects/bf040b2afd06492589c6028ef79a594d/users/6afb908a50d54be184a5b1e1f0430e6c/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:41.152937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:41.153333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:41.153753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.191546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:41.217052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] MFA Rules not processed for user `6afb908a50d54be184a5b1e1f0430e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:41.219818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:41.488621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 796/1636] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:41.492338 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:41.503262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:41.605670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51643611-b4b7-43cd-84c6-341e1e3b6eb5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:41.831188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:41.831809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:41.831809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:41.831809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:41.831809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.832515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:41.832515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:41.832717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abe481-52d2-4856-ac25-66aa4b771182 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:41.832717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:41.832717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:41.832717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:41.833932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.836961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 841/1637] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:28:41 2022] POST /identity/v3/projects => generated 387 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:28:41.841414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:41.892902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:41.897901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445bce1-94c8-4687-af63-5be9963b56b5 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:41.922017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 797/1638] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:41.940251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676862b-6f62-4e4f-88bb-f8977e41b961 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:41.947939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:41.948457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:41.948967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:41.949631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:41.950500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:41.952726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.037315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57513f39-92b0-431e-a174-023e8d060976 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.037866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57513f39-92b0-431e-a174-023e8d060976 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.037866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57513f39-92b0-431e-a174-023e8d060976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.055772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 842/1639] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:28:41 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:28:42.061683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.078776 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cc07b-9dad-425d-8cc8-f098f62bb7c9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.085318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.086000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-57513f39-92b0-431e-a174-023e8d060976 None None] MFA Rules not processed for user `a0a422ada9ab43609a7940a1df349b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:42.086541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.086960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.087385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.087882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-57513f39-92b0-431e-a174-023e8d060976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:42.088401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.090408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.107050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 843/1640] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:28:42.116190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.132916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:42.143904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fef75-86d7-4809-88d7-1f728d2972d0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.154291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.156371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.156634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.156911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] PATH_INFO: `/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0/users/922a0359e4cd4035b05ccb6c2cb08fd8/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.181507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3fb41e6df245f9a2d044eddf214cd0, user_id=922a0359e4cd4035b05ccb6c2cb08fd8, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.184686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bee88-bfd3-4600-a736-cde6d63751c0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.216643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 798/1641] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:42.218227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.218677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.218809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.243351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] MFA Rules not processed for user `a0a422ada9ab43609a7940a1df349b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:42.245062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:42.268057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 799/1642] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:42.286472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 844/1643] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:42 2022] PUT /identity/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0/users/922a0359e4cd4035b05ccb6c2cb08fd8/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:28:42.293035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.315028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61620f-9760-44f0-bfc9-4c377ff353bc None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.315412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61620f-9760-44f0-bfc9-4c377ff353bc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.315591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61620f-9760-44f0-bfc9-4c377ff353bc None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.317285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 845/1644] 10.222.0.42 () {54 vars in 961 bytes} [Fri May 27 16:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:42.327077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1953-3dca-435f-87bb-06ef6b8edaf7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.345267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.345597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.345881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.392111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:42.396681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:42.421503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.422436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.422861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.423413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.426821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.426821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.457083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 800/1645] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:28:42.458679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.472218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5adcff-3e22-42bf-9f24-d15467d4cdf0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.482209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.483351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.483699 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.490312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] PATH_INFO: `/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0/users/922a0359e4cd4035b05ccb6c2cb08fd8/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.490312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3fb41e6df245f9a2d044eddf214cd0, user_id=922a0359e4cd4035b05ccb6c2cb08fd8, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.494097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae31e-6b68-4351-8e8d-b4055ff77ebb admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.522084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 801/1646] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:28:42 2022] PUT /identity/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0/users/922a0359e4cd4035b05ccb6c2cb08fd8/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:28:42.531349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.531637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.532046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.567313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] MFA Rules not processed for user `922a0359e4cd4035b05ccb6c2cb08fd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:42.569424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:42.623040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 846/1647] 10.222.0.42 () {58 vars in 1058 bytes} [Fri May 27 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:42.624051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.657538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad55e9-809f-40c8-9319-a8a1532448ee None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.699216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:42.781733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 802/1648] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:42.789251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:42.835543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.836269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.836558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.836841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.853464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:42.907786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43521a-5231-4821-aeb0-54117fca1fd4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:42.912475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:42.913272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:42.913596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:42.913890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:42.927666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.935175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:42.936006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb50ae2-a63e-4a97-920f-489eff267aa5 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.939900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2f4a5-66ea-4b14-8eff-069e21c2d062 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:42.961000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 803/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:28:42.961322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 847/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:43.349887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:43.350762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:43.350762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:43.352362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 804/1651] 10.222.0.42 () {54 vars in 912 bytes} [Fri May 27 16:28:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:43.358740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8754e4-e985-4eaa-876a-9cee378bc56a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:43.359301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8754e4-e985-4eaa-876a-9cee378bc56a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:43.359802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8754e4-e985-4eaa-876a-9cee378bc56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:43.390307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-af8754e4-e985-4eaa-876a-9cee378bc56a None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:43.392147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-af8754e4-e985-4eaa-876a-9cee378bc56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:43.416204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 848/1652] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:28:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:43.424666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:43.426464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:43.513808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16189899-5c49-4ced-8267-137ee502d3d8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:43.519958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:43.521765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:43.522114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:43.522533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:43.534005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:43.535556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b0ad-58ed-4f7e-aa61-b7523152e6bd service glance] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:43.563061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 805/1653] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:44.276060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf498b-14c1-406f-b3bc-1d0e76d20614 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:44.276663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf498b-14c1-406f-b3bc-1d0e76d20614 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:44.276663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf498b-14c1-406f-b3bc-1d0e76d20614 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:44.278357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 849/1654] 10.222.0.42 () {54 vars in 912 bytes} [Fri May 27 16:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:44.284022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:44.285356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:44.285356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:44.391254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:44.394361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:44.549153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 806/1655] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:28:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:44.598468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf498b-14c1-406f-b3bc-1d0e76d20614 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:44.616055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf498b-14c1-406f-b3bc-1d0e76d20614 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:44.625407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:44.625995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:44.626285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:44.626566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:44.647790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:44.649929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f3e93-f01d-487b-9e55-f1fa29d7cdd9 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:44.673610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 850/1656] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:44.702361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:44.718727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e99ea-c358-4bcb-8cba-29014697407d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:44.728497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:44.728497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:44.728497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:44.728497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:44.844861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:44.846542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e450079-3e53-42e2-be78-2b3785f7a69e service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:44.870966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 807/1657] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:46.507434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c557-d585-465f-809d-65cf2f54652b None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:46.507813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c557-d585-465f-809d-65cf2f54652b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:46.508012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c557-d585-465f-809d-65cf2f54652b None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:46.516548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 851/1658] 10.222.0.42 () {54 vars in 960 bytes} [Fri May 27 16:28:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:28:46.520035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52814-42c1-410e-9692-835ed91e89d1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:46.520989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52814-42c1-410e-9692-835ed91e89d1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:46.521679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52814-42c1-410e-9692-835ed91e89d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:46.547064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:46.566580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-70c52814-42c1-410e-9692-835ed91e89d1 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:46.567827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-70c52814-42c1-410e-9692-835ed91e89d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:46.815997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c557-d585-465f-809d-65cf2f54652b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:46.820955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 808/1659] 10.222.0.42 () {58 vars in 1057 bytes} [Fri May 27 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:46.846147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c557-d585-465f-809d-65cf2f54652b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:46.860733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:46.861684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:46.862098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:46.862502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:46.870388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:46.913670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:46.916481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:46.988134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 852/1660] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:28:49.550312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359f7aa-fc47-4c16-9c97-d5e7618dd820 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:49.550935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359f7aa-fc47-4c16-9c97-d5e7618dd820 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:49.550935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359f7aa-fc47-4c16-9c97-d5e7618dd820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:49.604824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9359f7aa-fc47-4c16-9c97-d5e7618dd820 None None] MFA Rules not processed for user `391d41dab9294cd39d3b80d60570034e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:49.606743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9359f7aa-fc47-4c16-9c97-d5e7618dd820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:49.740306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 809/1661] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:49.745512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:49.767670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9505379-8f26-475c-a079-2ccdde930175 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:49.775579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:49.776546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:49.776836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:49.777120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:49.789838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:49.791876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:49.819785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 853/1662] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:53.833220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c339a-e670-476b-8052-ea7fb3188e96 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:53.836027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c339a-e670-476b-8052-ea7fb3188e96 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:53.840565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c339a-e670-476b-8052-ea7fb3188e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:53.920189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c19c339a-e670-476b-8052-ea7fb3188e96 None None] MFA Rules not processed for user `56799d191ffe4193918fa0221c27c0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:53.921452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c19c339a-e670-476b-8052-ea7fb3188e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:54.256303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 810/1663] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:54.274271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:54.294151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765131-e52c-4f72-9830-4bd001f858df service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:54.313564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:54.323190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:54.324508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:54.326487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:54.356613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:54.358869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a1610-4493-4f57-a50e-ae30ee0855e0 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:54.373859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 854/1664] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:55.394094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:55.395566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:55.396525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:55.430401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:55.432903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:55.489602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:55.600298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 811/1665] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:55.766076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:55.766807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:55.767098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:55.838131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] MFA Rules not processed for user `87d41932bfa94f369103ed29d29940b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:55.839397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:55.868007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:56.086511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 855/1666] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:56.105780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:56.138995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de044b-13f6-46a1-afd1-63ce57d059f1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:56.165678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:56.166376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:56.166670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:56.166965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:56.181648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:56.184825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:56.199039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:56.200527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:56.246996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 812/1667] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:56.469858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6642fe59-fa8f-4516-9989-685309efb2bc None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:56.482382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:56.483064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:56.483356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:56.483639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:56.531605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:56.535288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:56.577959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 856/1668] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:28:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:28:57.269685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:57.296131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d94cf6-97ea-4a92-aa2b-e52e29bcbeff service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:57.322279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:57.324288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:57.325599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:57.326577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:57.448650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:57.719248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:57.733215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3fe3f-42ac-4e52-988e-7272cc4ebf91 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:57.775056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 813/1669] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:57.805942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65248c-11d6-4364-a94a-0ddb4cd0189e service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:57.836927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:57.837868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:57.838324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:57.838700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] PATH_INFO: `/v3/users/ef7185d37eef4e48a716bbe8a8c39125` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:57.852908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7185d37eef4e48a716bbe8a8c39125)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:57.855002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:58.160573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:58.160971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:58.161355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:58.213411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:58.217952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:58.219210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:58.244360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 814/1670] 10.222.0.15 () {58 vars in 1015 bytes} [Fri May 27 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:58.251352 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7185d37eef4e48a716bbe8a8c39125'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:28:58.252029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7185d37eef4e48a716bbe8a8c39125'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:28:58.442663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:58.442663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c2e02-b7ae-494e-b2bd-72d92af88b1d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:58.442663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:58.442663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4183c-bd34-4465-8ef5-d8a0ac1f9c33 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:58.446154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 815/1671] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:28:58.461530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a300b7-39bb-4c17-97e1-5bc7c191cebc None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:58.462200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a300b7-39bb-4c17-97e1-5bc7c191cebc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:58.462663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a300b7-39bb-4c17-97e1-5bc7c191cebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:58.589942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-18a300b7-39bb-4c17-97e1-5bc7c191cebc None None] MFA Rules not processed for user `cf3e0adc32814585b57bf9f884840fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:58.589942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-18a300b7-39bb-4c17-97e1-5bc7c191cebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:58.752566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7185d37eef4e48a716bbe8a8c39125'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:28:58.753620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7185d37eef4e48a716bbe8a8c39125'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:28:58.795429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d2ce953e-447c-469f-80ad-a5c368ad1040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7185d37eef4e48a716bbe8a8c39125'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:28:58.846284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 857/1672] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:28:57 2022] DELETE /identity/v3/users/ef7185d37eef4e48a716bbe8a8c39125 => generated 0 bytes in 1409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:28:58.859357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:58.861551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:58.862009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:58.909449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:58.916713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:58.961995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 816/1673] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:58.965392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:58.965392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:58.965392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:59.046817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] MFA Rules not processed for user `cf3e0adc32814585b57bf9f884840fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:28:59.051873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:59.066827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 858/1674] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:28:59.152195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:28:59.232629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:59.234593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:59.261383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 817/1675] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:28:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:28:59.276393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:28:59.278722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:28:59.410829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4577e-525b-466b-8fa9-c7d34bae8fa2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:59.425756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:59.431919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:59.433977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:59.436843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:59.486765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:59.492626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1e8b8-bace-4fb7-9f3b-566a1744a7e6 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:59.524593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 859/1676] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:28:59.592605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4989d-7146-426c-b396-04a1ee795d8c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:28:59.612123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:28:59.612830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:28:59.613093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:28:59.613397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:28:59.640332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:28:59.642028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13becb-6609-461f-85fe-01ffc5bc4a57 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:28:59.680333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 818/1677] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:00.599766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4706cec-c4a7-4437-963f-79bb7f14117a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:00.600348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4706cec-c4a7-4437-963f-79bb7f14117a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:00.600436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4706cec-c4a7-4437-963f-79bb7f14117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:00.660723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b4706cec-c4a7-4437-963f-79bb7f14117a None None] MFA Rules not processed for user `391d41dab9294cd39d3b80d60570034e`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:00.671276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b4706cec-c4a7-4437-963f-79bb7f14117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:00.817712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 860/1678] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:01.474621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:01.475498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:01.475926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:01.479267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 819/1679] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:29:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:29:01.503869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad8fc-f382-4f13-939d-390cc6679410 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:01.505409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad8fc-f382-4f13-939d-390cc6679410 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:01.506065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad8fc-f382-4f13-939d-390cc6679410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:01.517216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:01.556619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-07bad8fc-f382-4f13-939d-390cc6679410 None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:01.562079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-07bad8fc-f382-4f13-939d-390cc6679410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:01.672749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 861/1680] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:29:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:03.289152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:03.292641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:03.334997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d753b-af91-4dfe-b3a4-05e412b15dcd None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:03.349001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:03.353000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:03.357105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:03.357424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] PATH_INFO: `/v3/projects/5dd9243059c8473c873d19c3a18d3f85` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:03.359551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd9243059c8473c873d19c3a18d3f85)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:03.363183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:03.480653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945777-0c89-453c-8902-93936ff42519 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:03.481213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945777-0c89-453c-8902-93936ff42519 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:03.481677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945777-0c89-453c-8902-93936ff42519 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:03.484523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 862/1681] 10.222.0.42 () {54 vars in 912 bytes} [Fri May 27 16:29:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:29:03.491801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:03.510079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:03.511209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:03.534199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd9243059c8473c873d19c3a18d3f85', 'request_id': 'req-e5ae085d-0434-445a-afb3-f0a333f164b2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:03.561520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:03.562966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:03.610190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd9243059c8473c873d19c3a18d3f85', 'request_id': 'req-e5ae085d-0434-445a-afb3-f0a333f164b2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:03.620320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd9243059c8473c873d19c3a18d3f85', 'request_id': 'req-e5ae085d-0434-445a-afb3-f0a333f164b2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:03.653302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] The token cache is being invalidate because project 5dd9243059c8473c873d19c3a18d3f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:03.654742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:03.655596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e5ae085d-0434-445a-afb3-f0a333f164b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:03.665203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 820/1682] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:03 2022] DELETE /identity/v3/projects/5dd9243059c8473c873d19c3a18d3f85 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:03.668753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee63a-d0b7-4c40-b877-f2b8ecee19e7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:03.669190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee63a-d0b7-4c40-b877-f2b8ecee19e7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:03.670413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ee63a-d0b7-4c40-b877-f2b8ecee19e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:03.690435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 863/1683] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:03.703940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:03.706069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:03.719994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06768796-7b7c-40fc-8c3a-523bb1293a13 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:03.742253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:03.743003 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:03.747975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:03.747975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:03.748194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-713ee63a-d0b7-4c40-b877-f2b8ecee19e7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:03.754911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-713ee63a-d0b7-4c40-b877-f2b8ecee19e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:03.764405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:03.767362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:03.782695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 864/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:29:03.784049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:03.786491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:03.910502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:04.032062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 821/1685] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:04.044063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.044571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.045684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.076096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6d0e9-3f53-4cc2-8283-a0915cacaf84 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:04.095323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:04.097425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:04.125454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:04.126142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.126263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 822/1686] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:04.126432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.126724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.132339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:04.133665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:04.179014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43270b-11e3-4e94-ad4d-4d5c40709215 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:04.198103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:04.198321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:04.198940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.199221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.199502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.200232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:04.200541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:04.201110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:04.220647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:04.221052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 865/1687] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:04.268823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 823/1688] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:04.274581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:04.303784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cbe97-50c6-4a4e-b679-321daec7ab65 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:04.316833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:04.317616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.317909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.318211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.318899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:04.321694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:04.505164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 866/1689] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:04 2022] POST /identity/v3/projects => generated 363 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:04.506458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:04.532850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7613a-feb0-4531-a7a6-4e4ffebaa517 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:04.768129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:04.773454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.773454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.773454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.773454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:04.778503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:04.781322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:04.792166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:04.808149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9ff6-3b2c-4620-b387-974630f351e9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:04.821221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:04.822634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:04.835057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:04.835599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] PATH_INFO: `/v3/users/3bd0a732c554439eb1434edc17e1b6dc` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:04.837911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd0a732c554439eb1434edc17e1b6dc)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:04.840259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:05.016783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 824/1690] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:04 2022] POST /identity/v3/users => generated 282 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:05.022677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:05.065098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab25a3-925b-4b2d-b141-0b7f15c1a293 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:05.088799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:05.099367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.100036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.101205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.102888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:05.107741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:05.141177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 825/1691] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:05.144960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:05.163903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90806809-2333-43fd-aad8-ee43d327ad42 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:05.186637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:05.187502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.188009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.192888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.195201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:05.200920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:05.252894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 826/1692] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:05.254448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:05.267078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd0a732c554439eb1434edc17e1b6dc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:05.270814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd0a732c554439eb1434edc17e1b6dc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:05.279705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:05.311532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84363440-ded9-4620-9acd-198e65d3e862 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:05.330725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:05.338585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.342892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.342892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] PATH_INFO: `/v3/projects/1a9389c75e1f44eb9fe3c9dbfc048058/users/f813f4cd34384dc68602017432ac2b13/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.397539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9389c75e1f44eb9fe3c9dbfc048058, user_id=f813f4cd34384dc68602017432ac2b13, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:05.403512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840e6f3-4105-4523-a61a-8fecd523fb60 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:05.436320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd0a732c554439eb1434edc17e1b6dc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:05.436320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd0a732c554439eb1434edc17e1b6dc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:05.515964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 827/1693] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:05 2022] PUT /identity/v3/projects/1a9389c75e1f44eb9fe3c9dbfc048058/users/f813f4cd34384dc68602017432ac2b13/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:05.519628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebbad-15a0-4808-b6d4-502ca992a368 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.519628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebbad-15a0-4808-b6d4-502ca992a368 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.519628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebbad-15a0-4808-b6d4-502ca992a368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.519981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ed99e0a2-fef2-420b-aa3c-f080813748dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd0a732c554439eb1434edc17e1b6dc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:05.584559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 867/1694] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:04 2022] DELETE /identity/v3/users/3bd0a732c554439eb1434edc17e1b6dc => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:05.588831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.589434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.591448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.599490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-639ebbad-15a0-4808-b6d4-502ca992a368 None None] MFA Rules not processed for user `cf3e0adc32814585b57bf9f884840fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:05.600751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-639ebbad-15a0-4808-b6d4-502ca992a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:05.764336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] MFA Rules not processed for user `f813f4cd34384dc68602017432ac2b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:05.769779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:05.882325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 828/1695] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:05.900802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:05.901530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:05.901817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:05.937213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 868/1696] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:05.938995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:05.940947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:05.970579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:05.971917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:06.142739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 829/1697] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:06.183055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937cc3b9-38c3-48c0-a18f-6b1584de6e62 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:06.200663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:06.201775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:06.202063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:06.202388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:06.204844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:06.212013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:06.220318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:06.222308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:06.355048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 869/1698] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:05 2022] POST /identity/v3/projects => generated 363 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:06.358952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:06.484909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474ba60-9ed4-4594-b14c-a05a51b09d1a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:06.537211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:06.668992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c5067-78ed-47ab-81e7-a50d419a29f3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:06.704919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:06.705711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:06.706005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:06.706408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:06.706984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:06.709193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:06.757183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:06.757897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:06.758222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:06.759429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:06.807490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 870/1699] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:06 2022] POST /identity/v3/users => generated 288 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:06.811942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:06.814891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:06.820513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:06.840156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c40b2-df98-4c38-80ae-bd584b8947fa admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:06.867316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:06.868719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:06.869112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:06.869460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:06.870276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:06.872022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ef808-66fe-41d4-9038-2b84b295b1b0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:06.881932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 830/1700] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:06.909160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 871/1701] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:06.919146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:06.964809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17341483-34bd-437a-8acc-dc0614afd0ba service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:07.005155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:07.011779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:07.012080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:07.012369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] PATH_INFO: `/v3/projects/8827041d1e8d4afe96530032280d2f90/users/2eb374cbf54f4ec6ba204365e113459c/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:07.026656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:07.031206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:07.031206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:07.090802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8827041d1e8d4afe96530032280d2f90, user_id=2eb374cbf54f4ec6ba204365e113459c, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:07.106030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] MFA Rules not processed for user `56799d191ffe4193918fa0221c27c0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:07.106200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:07.111430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:07.190355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 831/1702] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:06 2022] PUT /identity/v3/projects/8827041d1e8d4afe96530032280d2f90/users/2eb374cbf54f4ec6ba204365e113459c/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:07.194907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:07.379312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 872/1703] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:07.411028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:07.427245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091dd85-b3a5-41f7-af68-667c1cdeadb9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:07.433883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:07.434702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:07.435148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:07.435429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:07.436424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:07.447806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bed07c-d80f-4fb0-81d5-49851926bef2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:07.481142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:07.484137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 832/1704] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:07.505514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36076d3b-6c8a-4178-a3fb-56ed110aeaaf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:07.520066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:07.520066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:07.520066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:07.520066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] PATH_INFO: `/v3/projects/8827041d1e8d4afe96530032280d2f90/users/2eb374cbf54f4ec6ba204365e113459c/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:07.527496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8827041d1e8d4afe96530032280d2f90, user_id=2eb374cbf54f4ec6ba204365e113459c, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:07.536098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:07.567707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 873/1705] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:07 2022] PUT /identity/v3/projects/8827041d1e8d4afe96530032280d2f90/users/2eb374cbf54f4ec6ba204365e113459c/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:07.581250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e630911-7038-4cdc-9635-2c8f5606857c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:07.581835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e630911-7038-4cdc-9635-2c8f5606857c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:07.582222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e630911-7038-4cdc-9635-2c8f5606857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:07.632490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6e630911-7038-4cdc-9635-2c8f5606857c None None] MFA Rules not processed for user `2eb374cbf54f4ec6ba204365e113459c`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:07.634812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6e630911-7038-4cdc-9635-2c8f5606857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:07.645993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:07.874283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:07.876331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:07.887506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 833/1706] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:07 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:08.108109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccc15b-effd-408d-8d29-5b6b3ce6c503 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:08.119042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:08.122165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:08.122516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:08.122856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:08.152491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:08.152491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:08.208061 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 874/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:07 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:29:08.331065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5568-e738-4914-91fa-b14c56748191 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:08.331353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5568-e738-4914-91fa-b14c56748191 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:08.331633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5568-e738-4914-91fa-b14c56748191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:08.393432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-247c5568-e738-4914-91fa-b14c56748191 None None] MFA Rules not processed for user `87d41932bfa94f369103ed29d29940b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:08.396700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-247c5568-e738-4914-91fa-b14c56748191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:08.572968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 834/1708] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:09.022460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:09.025332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:09.157566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e2404-37fa-4723-ad38-3f64e8729523 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:09.164302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:09.166269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:09.167941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:09.167941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] PATH_INFO: `/v3/projects/5931bc4fc35642bdafa6391e2f99138c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:09.168713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5931bc4fc35642bdafa6391e2f99138c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:09.169824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: WARNING py.warnings [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:29:09.169824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: warnings.warn(msg) May 27 16:29:09.169824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: May 27 16:29:09.171045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:09.493792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5931bc4fc35642bdafa6391e2f99138c', 'request_id': 'req-04c7b654-df80-41e6-8835-8b8da3149efd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:09.512330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5931bc4fc35642bdafa6391e2f99138c', 'request_id': 'req-04c7b654-df80-41e6-8835-8b8da3149efd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:09.526278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:09.605229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5931bc4fc35642bdafa6391e2f99138c', 'request_id': 'req-04c7b654-df80-41e6-8835-8b8da3149efd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:09.665616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] The token cache is being invalidate because project 5931bc4fc35642bdafa6391e2f99138c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:09.665936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:09.673742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:09.685176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5568-e738-4914-91fa-b14c56748191 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:09.689200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 875/1709] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:09 2022] DELETE /identity/v3/projects/5931bc4fc35642bdafa6391e2f99138c => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:09.706581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5568-e738-4914-91fa-b14c56748191 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:10.008725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:10.018684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:10.020988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:10.022434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] PATH_INFO: `/v3/users/6afb908a50d54be184a5b1e1f0430e6c` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:10.027995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6afb908a50d54be184a5b1e1f0430e6c)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:10.034464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: WARNING py.warnings [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 16:29:10.034464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: warnings.warn(msg) May 27 16:29:10.034464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: May 27 16:29:10.036188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:10.057392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:10.126135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:10.128378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:10.185380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c7b654-df80-41e6-8835-8b8da3149efd admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:10.201844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:10.207587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:10.207587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:10.207587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] PATH_INFO: `/v3/users/f813f4cd34384dc68602017432ac2b13` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:10.210838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f813f4cd34384dc68602017432ac2b13)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:10.213653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:10.422756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6afb908a50d54be184a5b1e1f0430e6c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.451937 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6afb908a50d54be184a5b1e1f0430e6c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.487590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f813f4cd34384dc68602017432ac2b13'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.494890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f813f4cd34384dc68602017432ac2b13'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.550189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6afb908a50d54be184a5b1e1f0430e6c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.555969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6afb908a50d54be184a5b1e1f0430e6c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.555969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-58f64b2a-8d3e-4028-a499-818fa9a3cec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6afb908a50d54be184a5b1e1f0430e6c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.609137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 835/1710] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:09 2022] DELETE /identity/v3/users/6afb908a50d54be184a5b1e1f0430e6c => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:10.640067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f813f4cd34384dc68602017432ac2b13'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.641440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f813f4cd34384dc68602017432ac2b13'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.678795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f813f4cd34384dc68602017432ac2b13'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:10.703254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 876/1711] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:10 2022] DELETE /identity/v3/users/f813f4cd34384dc68602017432ac2b13 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:10.707938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:10.710751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:10.710751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:10.762223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:10.764845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:11.023095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 836/1712] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:11.097317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:11.098946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:11.253979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7c7a4-f631-4dfb-a589-a3cf00248fce admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:11.265072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:11.268288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:11.268816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:11.269271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:11.301920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:11.306569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:11.325291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:11.340014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 877/1713] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:12.105197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:12.107590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:12.143530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106177a3-7513-49d0-a5f1-b71c0d21d2d8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:12.177063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:12.177999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:12.178467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:12.178895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] PATH_INFO: `/v3/projects/bf040b2afd06492589c6028ef79a594d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:12.182832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf040b2afd06492589c6028ef79a594d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:12.188728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:12.462331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf040b2afd06492589c6028ef79a594d', 'request_id': 'req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:12.544505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf040b2afd06492589c6028ef79a594d', 'request_id': 'req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:12.551350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf040b2afd06492589c6028ef79a594d', 'request_id': 'req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:12.598505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] The token cache is being invalidate because project bf040b2afd06492589c6028ef79a594d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:12.599002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:12.601717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:12.623717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 837/1714] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:12 2022] DELETE /identity/v3/projects/bf040b2afd06492589c6028ef79a594d => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:12.623981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:12.641682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5b1e4-0d5d-4079-9faf-bbbd2f0b8cf0 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:12.844918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:12.846064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:12.846501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:12.847026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] PATH_INFO: `/v3/users/922a0359e4cd4035b05ccb6c2cb08fd8` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:12.849104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922a0359e4cd4035b05ccb6c2cb08fd8)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:12.851528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:13.117512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922a0359e4cd4035b05ccb6c2cb08fd8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:13.118891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922a0359e4cd4035b05ccb6c2cb08fd8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:13.213523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922a0359e4cd4035b05ccb6c2cb08fd8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:13.214218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922a0359e4cd4035b05ccb6c2cb08fd8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:13.242334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922a0359e4cd4035b05ccb6c2cb08fd8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:13.265504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 878/1715] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:12 2022] DELETE /identity/v3/users/922a0359e4cd4035b05ccb6c2cb08fd8 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:13.531011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:13.532875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:13.593790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaaebd-476b-4a6f-ae60-9a401062d1dd admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:13.615822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:13.750983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:13.752623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:13.753123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:13.753961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] PATH_INFO: `/v3/projects/1a9389c75e1f44eb9fe3c9dbfc048058` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:13.755763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9389c75e1f44eb9fe3c9dbfc048058)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:13.758002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:13.932603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9389c75e1f44eb9fe3c9dbfc048058', 'request_id': 'req-7bdea6e3-29fe-4e7a-b529-daee7b26711f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.072443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9389c75e1f44eb9fe3c9dbfc048058', 'request_id': 'req-7bdea6e3-29fe-4e7a-b529-daee7b26711f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.091559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9389c75e1f44eb9fe3c9dbfc048058', 'request_id': 'req-7bdea6e3-29fe-4e7a-b529-daee7b26711f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.101639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] The token cache is being invalidate because project 1a9389c75e1f44eb9fe3c9dbfc048058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:14.101934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.102625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.105500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 838/1716] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:13 2022] DELETE /identity/v3/projects/1a9389c75e1f44eb9fe3c9dbfc048058 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:14.110347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:14.112050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:14.142704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c96fa-bbd9-48e8-af39-b91759947467 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:14.247734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:14.253251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:14.255294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:14.255945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] PATH_INFO: `/v3/users/2eb374cbf54f4ec6ba204365e113459c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:14.268313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb374cbf54f4ec6ba204365e113459c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:14.268313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:14.318893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:14.475462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:14.477097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:14.507009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdea6e3-29fe-4e7a-b529-daee7b26711f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:14.530819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:14.531707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:14.532295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:14.533287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] PATH_INFO: `/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:14.537935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3fb41e6df245f9a2d044eddf214cd0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:14.555833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:14.576800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb374cbf54f4ec6ba204365e113459c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.576800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb374cbf54f4ec6ba204365e113459c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.696844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb374cbf54f4ec6ba204365e113459c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.698927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb374cbf54f4ec6ba204365e113459c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.724485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d0dc394-6a8a-4fdc-aa7c-bcbeaed5121f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb374cbf54f4ec6ba204365e113459c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:14.756074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 879/1717] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:14 2022] DELETE /identity/v3/users/2eb374cbf54f4ec6ba204365e113459c => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:14.956198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3fb41e6df245f9a2d044eddf214cd0', 'request_id': 'req-19c3adf7-4bac-4925-a711-678ed76d4a05'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:15.057053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3fb41e6df245f9a2d044eddf214cd0', 'request_id': 'req-19c3adf7-4bac-4925-a711-678ed76d4a05'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:15.063465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3fb41e6df245f9a2d044eddf214cd0', 'request_id': 'req-19c3adf7-4bac-4925-a711-678ed76d4a05'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:15.084956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] The token cache is being invalidate because project 6b3fb41e6df245f9a2d044eddf214cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:15.088514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:15.089400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-19c3adf7-4bac-4925-a711-678ed76d4a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:15.112978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 839/1718] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:14 2022] DELETE /identity/v3/projects/6b3fb41e6df245f9a2d044eddf214cd0 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:15.114594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.117229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.117229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.145688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:15.147159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:15.230993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 880/1719] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:15.250310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.250928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.251275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.282356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:15.284226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:15.314993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 840/1720] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:15.317817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.331835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a13a-cdb6-4280-886d-bfe93d021f99 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.340993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.342019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.342376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.342725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.343748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:15.343964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.345949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.361435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 881/1721] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:15.365833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.376637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda5e86-cebe-4cd3-86bb-dbb1ace8f8e3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.383721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.386002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.386462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.387235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.387972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.391005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.409080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:15.508829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 841/1722] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:15 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:15.511029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.522586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9f5c2-4564-49cf-af8f-7143366f02f6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.627161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.629024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.629645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.639334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.639334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.639334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.745696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 882/1723] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:15 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:15.747497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.758135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65608044-882f-4791-817d-e7196d1e3e22 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.765521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.769710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.770257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.770758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.771703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.773945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.793608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 842/1724] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:15.797808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.809720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b756c-bd1f-4ca1-a87e-f7783e8e46d9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.817126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.818058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.818602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.818897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.819645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.821424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.833681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 883/1725] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:29:15.838907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.849221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a3e54-fb81-4e99-9416-e47389f8c0ce admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.857232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.858210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.858482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.858759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] PATH_INFO: `/v3/projects/79e1c11b884d4a368f3b27940d577a9f/users/eb25de33cfc845c6b79c5458f2f99fbd/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.876713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e1c11b884d4a368f3b27940d577a9f, user_id=eb25de33cfc845c6b79c5458f2f99fbd, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.879523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c987876-a2fc-4dae-a57a-8b50471f2e1f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.934050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:15.936791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:15.954433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446297-20ee-4f2b-9c42-5ee217f60747 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:15.960892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:15.961799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.962083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:15.962902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] PATH_INFO: `/v3/projects/8827041d1e8d4afe96530032280d2f90` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:15.965375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8827041d1e8d4afe96530032280d2f90)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:15.967495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:15.980649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 843/1726] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:15 2022] PUT /identity/v3/projects/79e1c11b884d4a368f3b27940d577a9f/users/eb25de33cfc845c6b79c5458f2f99fbd/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:15.999140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333395b-0818-4964-b74d-ad429482f99c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:15.999429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333395b-0818-4964-b74d-ad429482f99c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.002202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333395b-0818-4964-b74d-ad429482f99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.038819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6333395b-0818-4964-b74d-ad429482f99c None None] MFA Rules not processed for user `eb25de33cfc845c6b79c5458f2f99fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:16.040105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6333395b-0818-4964-b74d-ad429482f99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:16.116327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 844/1727] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:16.120930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333395b-0818-4964-b74d-ad429482f99c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.124692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8827041d1e8d4afe96530032280d2f90', 'request_id': 'req-84248763-88a5-4e92-a36d-0e3f69338f43'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:16.132578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8827041d1e8d4afe96530032280d2f90', 'request_id': 'req-84248763-88a5-4e92-a36d-0e3f69338f43'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:16.224082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8827041d1e8d4afe96530032280d2f90', 'request_id': 'req-84248763-88a5-4e92-a36d-0e3f69338f43'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:16.240415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] The token cache is being invalidate because project 8827041d1e8d4afe96530032280d2f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:16.241132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:16.242004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-84248763-88a5-4e92-a36d-0e3f69338f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:16.249459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 884/1728] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:15 2022] DELETE /identity/v3/projects/8827041d1e8d4afe96530032280d2f90 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:16.251688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b652863-f961-4ba4-8b50-8b61b49019e8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.252323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b652863-f961-4ba4-8b50-8b61b49019e8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.253077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b652863-f961-4ba4-8b50-8b61b49019e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.258444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333395b-0818-4964-b74d-ad429482f99c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.274686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:16.277973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3b652863-f961-4ba4-8b50-8b61b49019e8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:16.279690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3b652863-f961-4ba4-8b50-8b61b49019e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:16.368010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 885/1729] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:16.381751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.382074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.382410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.421762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:16.424949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:16.434360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 886/1730] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:16.447617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.477590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff073-0767-4066-9998-fb933cf31314 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.483641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:16.483641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.483641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.483641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.484030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72589-970b-42dd-8490-82109821349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:16.484282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:16.484639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72589-970b-42dd-8490-82109821349a admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.484966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72589-970b-42dd-8490-82109821349a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.485248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b72589-970b-42dd-8490-82109821349a admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.486028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72589-970b-42dd-8490-82109821349a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:16.486340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72589-970b-42dd-8490-82109821349a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:16.487287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:16.488080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b72589-970b-42dd-8490-82109821349a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:16.508765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 887/1731] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:16.510568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72589-970b-42dd-8490-82109821349a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.521251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b72589-970b-42dd-8490-82109821349a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.526640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:16.527300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.527585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.528238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.528960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:16.532844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:16.598036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 845/1732] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/projects => generated 397 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:16.606392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.612782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:16.664007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 888/1733] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:16.665444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.779138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab64be-bb3f-4f1e-af3f-9c7d18bf95ce admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.788644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:16.789716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.790192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.791570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.792450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:16.794452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f91ec-7a6c-4b9d-946a-43f8197790e1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.795113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7d058-97e4-442b-a101-069949ff4594 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:16.812253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:16.812959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.813451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.813731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.815523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:16.817618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:16.964015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 846/1734] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/users => generated 305 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:16.966317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61491d8-cf87-4d33-85e1-4a148973248e None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:16.966787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61491d8-cf87-4d33-85e1-4a148973248e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:16.967393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61491d8-cf87-4d33-85e1-4a148973248e None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:16.968654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 889/1735] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:16 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:16.972179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.974099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 847/1736] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:29:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:29:16.980429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61491d8-cf87-4d33-85e1-4a148973248e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:16.989324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb294-6507-4f9f-9d58-0296f2471a60 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:16.996177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.002311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.002311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.002311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.002311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.005855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f5af0b-1261-4e7f-873a-c5a9b1a1f84d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.019902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61491d8-cf87-4d33-85e1-4a148973248e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.032894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.035245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 890/1737] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:17.036689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.036988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.037270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.048796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.049189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.053473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.054344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.061814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.118955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:17.125254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 848/1738] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:17.126194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:17.126663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.136894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:17.143609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cdb06-df3f-40f8-a664-3350799ae986 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.156492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.167274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.167826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.168295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] PATH_INFO: `/v3/projects/bd30733430b04fdf820b98d351a7bc77/users/36defdb2d69e403b9567e05189de72f1/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.200307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd30733430b04fdf820b98d351a7bc77, user_id=36defdb2d69e403b9567e05189de72f1, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.203323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.310564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 849/1739] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:17 2022] PUT /identity/v3/projects/bd30733430b04fdf820b98d351a7bc77/users/36defdb2d69e403b9567e05189de72f1/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:17.319340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.319721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 891/1740] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:29:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:17.323101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.344265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27374dc1-6159-4654-9c1a-79db77f23f1b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.450037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.451263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.451556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.451835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.452631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.455115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d61ca0-b35c-4796-acbb-6572cf16b7ce admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.455883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.461000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.462435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.465750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.466393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.467310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.469142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.482859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 850/1741] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:17.482981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 892/1742] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:17.486554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.487535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.496755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:17.500581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea6aba-0acb-49bb-b049-a8a9d04f5884 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.509403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.511665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.513396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.514091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] PATH_INFO: `/v3/projects/8fe16eafd7cb412d941d0e41b1013151/users/f593088ec7b34d209402ea326d80d47b/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.540382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe16eafd7cb412d941d0e41b1013151, user_id=f593088ec7b34d209402ea326d80d47b, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.544907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.616377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a84024-66b0-45f4-b9dc-cb12ec853010 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.620524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 851/1743] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:17 2022] PUT /identity/v3/projects/8fe16eafd7cb412d941d0e41b1013151/users/f593088ec7b34d209402ea326d80d47b/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:17.622156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:17.752505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.752801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.753193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.754806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.761685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa49c80-836c-4d1f-bee6-2a6969c60d13 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:17.773308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.775424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:17.776170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.776553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.776948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] PATH_INFO: `/v3/projects/bd30733430b04fdf820b98d351a7bc77/users/36defdb2d69e403b9567e05189de72f1/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.782409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd30733430b04fdf820b98d351a7bc77, user_id=36defdb2d69e403b9567e05189de72f1, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:17.783915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18124857-e669-4c26-832c-e9a5d26dd0f3 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.786887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c4644-b74d-426e-874c-159d108cd2fc admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:17.809849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 893/1744] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:17.825553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.826494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.826877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.844407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 852/1745] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:17 2022] PUT /identity/v3/projects/bd30733430b04fdf820b98d351a7bc77/users/36defdb2d69e403b9567e05189de72f1/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:17.850684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:17.850992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:17.851255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:17.905970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] MFA Rules not processed for user `f593088ec7b34d209402ea326d80d47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:17.907474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:17.924746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:17.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] MFA Rules not processed for user `36defdb2d69e403b9567e05189de72f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:17.944378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:18.047109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 894/1746] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:18.051301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:18.135694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 853/1747] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:18.147541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:18.149175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:18.264169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:18.284233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a722bea-c043-40b9-b575-d20b620be8d9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:18.300573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:18.300573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:18.300573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:18.300573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:18.301683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:18.302279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3689a4-2f41-4530-a60c-9c64fead189e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:18.304283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:18.314929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:18.315719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:18.316077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:18.316433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:18.337007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:18.340700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:18.354331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 854/1748] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:18.443468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 895/1749] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:18 2022] POST /identity/v3/projects => generated 365 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:18.445056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:18.458720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87100cbe-1187-4134-80d7-38ccc7253435 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:18.567792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:18.568619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:18.568912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:18.569191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:18.574332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:18.574332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:18.643685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:18.789462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528b925-563a-4b2c-958c-8d0368a7039e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:18.789462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:18.789462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:18.789462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:18.789791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:18.789791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:18.862249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:18.869572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:18.890301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 896/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:19.286357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 855/1751] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:18 2022] POST /identity/v3/users => generated 289 bytes in 837 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:19.287311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:19.297308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0567c-40ec-4a7a-b31f-412ce1555c91 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:19.305798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:19.306487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.306821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.307759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.308503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:19.310175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:19.320580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 897/1752] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:19.326517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:19.344479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc67ba-a353-420f-9d30-abf9ab92843c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:19.349478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:19.350385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.350673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.350956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] PATH_INFO: `/v3/projects/0b540ff750f442aa9de5211f4ab6f31d/users/0dc24eb3d6174181b5e033872bf3a295/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.367944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b540ff750f442aa9de5211f4ab6f31d, user_id=0dc24eb3d6174181b5e033872bf3a295, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:19.370733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91fed2-8f30-41ce-8e75-e99442004625 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:19.697902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 856/1753] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:19 2022] PUT /identity/v3/projects/0b540ff750f442aa9de5211f4ab6f31d/users/0dc24eb3d6174181b5e033872bf3a295/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:19.701316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:19.714138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e0e6a-1b96-4959-97a4-c9ccd35dbc6a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:19.847558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:19.848267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.848534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.848808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.849569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:19.851298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc309c-4389-40bb-907a-222e60c34a5d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:19.872274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e00277-523e-4496-bab4-f3ab486e5252 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.873105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e00277-523e-4496-bab4-f3ab486e5252 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.875754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e00277-523e-4496-bab4-f3ab486e5252 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.875754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 857/1754] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:29:19.876467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 898/1755] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:19.881442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e00277-523e-4496-bab4-f3ab486e5252 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:19.889910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.890034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.890706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.914397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e00277-523e-4496-bab4-f3ab486e5252 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:19.924990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:19.925551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.925892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.926170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] PATH_INFO: `/v3/projects/0b540ff750f442aa9de5211f4ab6f31d/users/0dc24eb3d6174181b5e033872bf3a295/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:19.928069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:19.937548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:19.939094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b540ff750f442aa9de5211f4ab6f31d, user_id=0dc24eb3d6174181b5e033872bf3a295, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:19.947285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42139e6-3f2b-47ae-94fd-237e94c047fe admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:19.991188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 858/1756] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:19 2022] PUT /identity/v3/projects/0b540ff750f442aa9de5211f4ab6f31d/users/0dc24eb3d6174181b5e033872bf3a295/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:19.992631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:19.992922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:19.996242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:20.051357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] MFA Rules not processed for user `0dc24eb3d6174181b5e033872bf3a295`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:20.057394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:20.117539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 899/1757] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:29:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:20.220238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 859/1758] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:20.233231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:20.234690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:20.273289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:20.380809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc823df-2df4-4166-8c27-88a48083d692 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:20.388565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:20.389241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:20.389553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:20.389831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:20.401434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:20.403169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:20.404079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502179b-24d2-4152-8d3b-eadb6da87219 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:20.419328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 860/1759] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:20.499155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:20.499834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:20.500119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:20.500394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:20.517966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:20.520695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:20.527767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:20.535897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 900/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:20.799484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:20.809948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bb198-4d89-4aa2-817d-50beecfd95d4 service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:20.813901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:20.814733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:20.815015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:20.815292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:20.891567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:20.893386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812db70c-f610-4ed5-b01c-636995b2ada2 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:20.908039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 861/1761] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:20.961076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:21.698057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:21.909835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:22.377957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:27.280092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:27.353963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fedaa-9d4b-48db-847c-e772cf395891 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:27.359059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:27.371136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:27.371136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:27.371136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] PATH_INFO: `/v3/users/eb25de33cfc845c6b79c5458f2f99fbd` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:27.373962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb25de33cfc845c6b79c5458f2f99fbd)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:27.375982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:27.416332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:27.593331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb25de33cfc845c6b79c5458f2f99fbd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:27.596239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb25de33cfc845c6b79c5458f2f99fbd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:27.670163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb25de33cfc845c6b79c5458f2f99fbd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:27.670299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb25de33cfc845c6b79c5458f2f99fbd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:27.702105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb25de33cfc845c6b79c5458f2f99fbd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:27.725106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 901/1762] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:27 2022] DELETE /identity/v3/users/eb25de33cfc845c6b79c5458f2f99fbd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:27.730402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:27.731026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:27.731557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:27.759399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:27.760520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:27.845741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 862/1763] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:27.855565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:27.857689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:27.923384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f485c-236b-472b-9f0c-eb10794299b8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:27.927085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:27.927818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:27.928115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:27.928476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:27.943854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:27.946673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:27.958455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 902/1764] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:28.351381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:28.353500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:28.369756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce8b79-6372-4539-b2f4-543f59a2042f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:28.375848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:28.376336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:28.379745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:28.379745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] PATH_INFO: `/v3/projects/79e1c11b884d4a368f3b27940d577a9f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:28.379745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e1c11b884d4a368f3b27940d577a9f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:28.380486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:28.487420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e1c11b884d4a368f3b27940d577a9f', 'request_id': 'req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.555402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e1c11b884d4a368f3b27940d577a9f', 'request_id': 'req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.562563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e1c11b884d4a368f3b27940d577a9f', 'request_id': 'req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.574835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] The token cache is being invalidate because project 79e1c11b884d4a368f3b27940d577a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:28.575114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.580221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.580221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 863/1765] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:28 2022] DELETE /identity/v3/projects/79e1c11b884d4a368f3b27940d577a9f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:28.581262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:28.591398 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c2fa-91b0-46e5-a3c3-72f9585b9170 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:28.652223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:28.653109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:28.653435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:28.653721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] PATH_INFO: `/v3/users/36defdb2d69e403b9567e05189de72f1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:28.654914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36defdb2d69e403b9567e05189de72f1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:28.656864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:28.832173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:28.835810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36defdb2d69e403b9567e05189de72f1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.835810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36defdb2d69e403b9567e05189de72f1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.949830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36defdb2d69e403b9567e05189de72f1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.950452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36defdb2d69e403b9567e05189de72f1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.963243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0999c9ad-a150-4e05-98d6-4dc7bd0b254c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36defdb2d69e403b9567e05189de72f1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:28.975268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 903/1766] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:28 2022] DELETE /identity/v3/users/36defdb2d69e403b9567e05189de72f1 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:29.448297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:29.450195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:29.465965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11003a6-a26a-418c-9b1d-74e0b8c47ac9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:29.477517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:29.542946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:29.544487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:29.545221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:29.545629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] PATH_INFO: `/v3/projects/bd30733430b04fdf820b98d351a7bc77` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:29.546996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd30733430b04fdf820b98d351a7bc77)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:29.549227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:29.699947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd30733430b04fdf820b98d351a7bc77', 'request_id': 'req-d5299eff-a25b-4a10-8424-c6083fa7a013'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:29.767207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd30733430b04fdf820b98d351a7bc77', 'request_id': 'req-d5299eff-a25b-4a10-8424-c6083fa7a013'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:29.775219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd30733430b04fdf820b98d351a7bc77', 'request_id': 'req-d5299eff-a25b-4a10-8424-c6083fa7a013'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:29.789321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] The token cache is being invalidate because project bd30733430b04fdf820b98d351a7bc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:29.789514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:29.790235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d5299eff-a25b-4a10-8424-c6083fa7a013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:29.792500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 864/1767] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:29 2022] DELETE /identity/v3/projects/bd30733430b04fdf820b98d351a7bc77 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:29.796860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:29.797313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:29.797445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:29.818069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:29.819316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:29.900764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 904/1768] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:29.910368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:29.910467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:29.910811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:29.940978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:29.942430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:29.951938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 865/1769] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:29.960245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:29.970779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bb8bf-0ffd-4f08-ab53-50af6f3f30b3 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:29.975500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:29.976159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:29.976450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:29.976729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:29.977492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:29.977807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:29.979426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:29.991405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 905/1770] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:29.996528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.005605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed980d-6000-43f3-9c7f-919b1f886539 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.010129 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.010772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.011769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.012296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.012728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.014663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.099656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 866/1771] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:29 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:30.104096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.114951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b004021-cd21-437e-bba9-d3cbc03642a7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.171670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.172334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.172627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.172901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.173629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.175551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.248637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 906/1772] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:30 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:30.252314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.263911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795177a1-0278-4006-8499-22e4071ae08e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.268689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.269368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.269663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.269944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.270672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.272283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.283409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 867/1773] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:29:30.287357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.297761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09134c45-f5b8-4710-8892-df5519e2c829 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.302269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.303031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.303321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.303606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.304358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.306009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c8cfd-208d-4562-8d6e-f8d64c86db5b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.321486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.321603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 907/1774] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:30.332385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e36cd-ef8e-4a52-b5be-2f10e947ef3c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.337430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.338292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.338585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.338865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] PATH_INFO: `/v3/projects/ae3f652185954fba813a3bf40224c506/users/99620f411a1540f88c045894b4fad922/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.354971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3f652185954fba813a3bf40224c506, user_id=99620f411a1540f88c045894b4fad922, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.357946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.420346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 868/1775] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:30 2022] PUT /identity/v3/projects/ae3f652185954fba813a3bf40224c506/users/99620f411a1540f88c045894b4fad922/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:30.431159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.431553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.432011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.461617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] MFA Rules not processed for user `99620f411a1540f88c045894b4fad922`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:30.462840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:30.482290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:30.533502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 908/1776] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:30.535534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.580433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:30.599376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf78de-1ac3-4855-b9fa-33f03f3bbe40 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.604265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.605709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.606005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.610106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.610106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.610106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.712056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 869/1777] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:30 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:30.716448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.740063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02053c8d-720c-4491-9116-0355a76c8f06 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.802117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.802746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.802975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.803251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.803960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.806123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:30.958779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 909/1778] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:30 2022] POST /identity/v3/users => generated 298 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:30.962390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:30.978846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616fa7d-3dc7-45ce-8d56-4ab726c2d73d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:30.985980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:30.986659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:30.986948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:30.987224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:30.987950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:30.989759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.003292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 870/1779] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:31.007637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:31.018848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccfce1-6f9c-4024-bc1f-1d8a4fe470eb admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:31.023715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:31.024557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.024843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.025118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] PATH_INFO: `/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9/users/b31453db638d4973ae2e011cebfe9e77/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.043841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233d2fe0e03b4cf6878f23bcfc30c9c9, user_id=b31453db638d4973ae2e011cebfe9e77, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:31.046684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.116206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 910/1780] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:31 2022] PUT /identity/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9/users/b31453db638d4973ae2e011cebfe9e77/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:31.120113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:31.130532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a09675-969e-40ce-b173-3e0968cfc429 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:31.187027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:31.187722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.188009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.188293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.189056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:31.190738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262067ef-403c-4b5c-b86a-3e5c6ab6a795 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.201791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 871/1781] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:31.206328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:31.217914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e9764-26e4-4db0-bc7e-e8c2f6472fe4 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:31.222484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:31.223366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.223643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.223916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] PATH_INFO: `/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9/users/b31453db638d4973ae2e011cebfe9e77/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.225923 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233d2fe0e03b4cf6878f23bcfc30c9c9, user_id=b31453db638d4973ae2e011cebfe9e77, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:31.228561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.239853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 911/1782] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:31 2022] PUT /identity/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9/users/b31453db638d4973ae2e011cebfe9e77/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:31.250520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.251647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.251647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.280964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] MFA Rules not processed for user `b31453db638d4973ae2e011cebfe9e77`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:31.282357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:31.354133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 872/1783] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:31.366404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:31.368113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:31.427207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0fa8b-378c-4a4e-a3c6-155a77046cd7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:31.430780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:31.431418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.431724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.432000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.441059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:31.442533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.452495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 912/1784] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:31.599457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:31.614184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6fd23-2c6b-419c-9d7b-011a65ce8eb9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:31.618599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:31.619503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:31.619938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:31.620364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:31.676169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:31.709421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:31.711128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea62627-4fd7-4930-b807-5748e07de490 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:31.722686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 873/1785] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:32.767462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:32.789076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:32.876891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eddf8-dbb6-46e8-ad10-da66ecd80816 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:32.883001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:32.884741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:32.884741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:32.884741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] PATH_INFO: `/v3/users/99620f411a1540f88c045894b4fad922` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:32.887452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99620f411a1540f88c045894b4fad922)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:32.888988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:33.060118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99620f411a1540f88c045894b4fad922'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:33.061640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99620f411a1540f88c045894b4fad922'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:33.146057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99620f411a1540f88c045894b4fad922'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:33.147441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99620f411a1540f88c045894b4fad922'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:33.167321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99620f411a1540f88c045894b4fad922'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:33.194964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 913/1786] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:32 2022] DELETE /identity/v3/users/99620f411a1540f88c045894b4fad922 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:33.199862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:33.203987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:33.203987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:33.246388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:33.247871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:33.358018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 874/1787] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:33.370544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:33.373942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:33.437787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875065-1ac2-43f1-91bb-80f1190b21b2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:33.441279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:33.442112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:33.442394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:33.442665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:33.452462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:33.453937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:33.463299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 914/1788] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:34.159032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:34.160567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:34.173110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924f0a-31ee-427d-a083-d703cac4e56b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:34.177888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:34.178772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:34.179187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:34.179468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] PATH_INFO: `/v3/projects/ae3f652185954fba813a3bf40224c506` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:34.180692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3f652185954fba813a3bf40224c506)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:34.182635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:34.304813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3f652185954fba813a3bf40224c506', 'request_id': 'req-d8bedfa1-f279-4730-9527-d75f0745bb2b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.391327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3f652185954fba813a3bf40224c506', 'request_id': 'req-d8bedfa1-f279-4730-9527-d75f0745bb2b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.412694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3f652185954fba813a3bf40224c506', 'request_id': 'req-d8bedfa1-f279-4730-9527-d75f0745bb2b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.426019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] The token cache is being invalidate because project ae3f652185954fba813a3bf40224c506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:34.426677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.427355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.428981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 875/1789] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:34 2022] DELETE /identity/v3/projects/ae3f652185954fba813a3bf40224c506 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:34.433081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:34.443577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89dd05-14ab-4ac1-9b1c-899a60fa79aa service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:34.484672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:34.514287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:34.515362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:34.515847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:34.516148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] PATH_INFO: `/v3/users/b31453db638d4973ae2e011cebfe9e77` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:34.517435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31453db638d4973ae2e011cebfe9e77)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:34.519461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:34.739259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31453db638d4973ae2e011cebfe9e77'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.739968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31453db638d4973ae2e011cebfe9e77'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.827637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31453db638d4973ae2e011cebfe9e77'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.837180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31453db638d4973ae2e011cebfe9e77'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.853094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-bbae5125-c1d7-42e9-8e64-4ab4c5d31702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31453db638d4973ae2e011cebfe9e77'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:34.874113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 915/1790] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:34 2022] DELETE /identity/v3/users/b31453db638d4973ae2e011cebfe9e77 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:35.432823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:35.441401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:35.448886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bedfa1-f279-4730-9527-d75f0745bb2b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:35.514674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:35.520678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:35.522007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:35.522343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:35.522679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] PATH_INFO: `/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:35.524159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233d2fe0e03b4cf6878f23bcfc30c9c9)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:35.526217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:35.685208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233d2fe0e03b4cf6878f23bcfc30c9c9', 'request_id': 'req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:35.751373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233d2fe0e03b4cf6878f23bcfc30c9c9', 'request_id': 'req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:35.758945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233d2fe0e03b4cf6878f23bcfc30c9c9', 'request_id': 'req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:35.777491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] The token cache is being invalidate because project 233d2fe0e03b4cf6878f23bcfc30c9c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:35.778031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:35.778814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0b8e63f2-5100-4576-aa8e-93ef5994f9e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:35.780914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 876/1791] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:35 2022] DELETE /identity/v3/projects/233d2fe0e03b4cf6878f23bcfc30c9c9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:35.786588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:35.787755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:35.788174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:35.811427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:35.813928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:35.904211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 916/1792] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:35.912031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:35.912249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:35.912517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:35.925311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:35.930875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:35.946125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:35.947079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:35.961994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 877/1793] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:35.962190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e298-58d4-4d70-a46a-dcca4594a8fd None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:35.964083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:35.976932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea6316-350f-4e43-9ce0-d99ede2c782a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:35.994459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:35.995828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:35.996880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.002008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] PATH_INFO: `/v3/users/f593088ec7b34d209402ea326d80d47b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.006677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f593088ec7b34d209402ea326d80d47b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.014264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.014264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.014264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.014264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.014816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:36.014816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.017096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.018537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.036344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 878/1794] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:36.040580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.055510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39495f-8477-4f18-a04e-0fb516b3a1b3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.064640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.065640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.065762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.066114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.066708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.077501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.159261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 879/1795] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:36 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:36.163401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.187715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f593088ec7b34d209402ea326d80d47b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:36.189370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f593088ec7b34d209402ea326d80d47b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:36.194280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:36.267511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5ad4b-6b89-4551-bb80-c42ef03d5f01 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.275804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.278301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.278958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.279288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f593088ec7b34d209402ea326d80d47b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:36.280250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f593088ec7b34d209402ea326d80d47b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:36.291060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.291060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.291530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.304292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d5e5d68c-a194-4f14-bba0-d25fed7ce1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f593088ec7b34d209402ea326d80d47b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:36.319988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 917/1796] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:35 2022] DELETE /identity/v3/users/f593088ec7b34d209402ea326d80d47b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:36.324787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.325520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.325867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.345318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:36.347456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:36.399170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 880/1797] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:36 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:36.405332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.407339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:36.421956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eb820-29ec-468d-bf2a-42e1fb8989f1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.427218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.427921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.428209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.431823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 918/1798] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:36.432035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.432035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.432035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.439581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.441107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:36.446123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 881/1799] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:36.450298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.462246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fd744-224a-46d7-96a2-fb4f598a3ad0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.470179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.474581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.474581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.474581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.474581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.474943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.490063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 882/1800] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:29:36.512586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.538504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8835a-3ea3-4e1e-959b-b6c4952030c9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.547883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c3cc4-09f6-4ca1-9791-e6afd449f33b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.548361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.549481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.549771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.550366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] PATH_INFO: `/v3/projects/8cb36db936a740b7aa7c0e049917d608/users/607c44df3e30476d985e898762548d92/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.553809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.559389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.559389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.559389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.567796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:36.572951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.573598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb36db936a740b7aa7c0e049917d608, user_id=607c44df3e30476d985e898762548d92, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:36.575632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288ca0b-b96b-4180-82a4-a9d21c3c65ac service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.577239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:36.592452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 919/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:29:36.709921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 883/1802] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:36 2022] PUT /identity/v3/projects/8cb36db936a740b7aa7c0e049917d608/users/607c44df3e30476d985e898762548d92/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:36.711194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.711495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.711772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.741838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] MFA Rules not processed for user `607c44df3e30476d985e898762548d92`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:36.743130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:36.832600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:36.837933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 920/1803] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:36.843174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:36.974621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2986a2e-0cfc-4572-b05c-cf21c258c8ed admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:36.990525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:36.991252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:36.991542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:36.995864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:36.996764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.004227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.164855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 884/1804] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:36 2022] POST /identity/v3/projects => generated 373 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:37.167117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.181817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e101c3-3dc1-4ee0-92eb-a9375e895eda None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.295276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.297623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:37.332664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.333552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.333917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:37.334275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:37.335719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.338144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.456120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 921/1805] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:37 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:37.460230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.463969 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d41b73-331c-40c7-b24a-dc63bdebd1fd admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.473008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.475026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.475485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:37.475909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] PATH_INFO: `/v3/projects/8fe16eafd7cb412d941d0e41b1013151` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:37.478078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe16eafd7cb412d941d0e41b1013151)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.483002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.494699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33382e8c-2fad-4f8a-a535-31243d56aad1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.500714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.501561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.501887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:37.502349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:37.503161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.504924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.526247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 922/1806] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:37.531049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.550797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95f1bd-9021-4bed-b60b-1bcb72f98d42 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.555685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.556827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.557221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:37.557751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] PATH_INFO: `/v3/projects/4a48e0346ab344b7a3267458b812c5da/users/b08a238573504868ba12dc5beb50f700/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:37.581548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a48e0346ab344b7a3267458b812c5da, user_id=b08a238573504868ba12dc5beb50f700, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.586117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.689530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 923/1807] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:37 2022] PUT /identity/v3/projects/4a48e0346ab344b7a3267458b812c5da/users/b08a238573504868ba12dc5beb50f700/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:37.693868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.706794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe16eafd7cb412d941d0e41b1013151', 'request_id': 'req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:37.788716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe16eafd7cb412d941d0e41b1013151', 'request_id': 'req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:37.792228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5fbd-ffff-46e4-a8d6-aaafcd40d88b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.812208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe16eafd7cb412d941d0e41b1013151', 'request_id': 'req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:37.835334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] The token cache is being invalidate because project 8fe16eafd7cb412d941d0e41b1013151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:37.835702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:37.836824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:37.843602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 885/1808] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:37 2022] DELETE /identity/v3/projects/8fe16eafd7cb412d941d0e41b1013151 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:37.844343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.846219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:37.853723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:37.880145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.881020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.881489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:37.881660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:37.882379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:37.884033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:37.886525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:37.917529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 924/1809] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:37.937149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:37.946808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:37.976722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44772403-3873-4dea-be07-42ad5c1ca453 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.992251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2f5fd-10e8-4018-b10f-4ffcc69d7a7b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:37.993901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:37.998582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:37.999907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:38.000964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] PATH_INFO: `/v3/projects/4a48e0346ab344b7a3267458b812c5da/users/b08a238573504868ba12dc5beb50f700/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:38.007024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:38.007247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a48e0346ab344b7a3267458b812c5da, user_id=b08a238573504868ba12dc5beb50f700, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:38.007949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:38.008246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:38.010631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c7fc5-40b6-454d-ac1c-0d338e882dc9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:38.012749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] PATH_INFO: `/v3/users/0dc24eb3d6174181b5e033872bf3a295` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:38.014480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc24eb3d6174181b5e033872bf3a295)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:38.018163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:38.031751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 925/1810] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:37 2022] PUT /identity/v3/projects/4a48e0346ab344b7a3267458b812c5da/users/b08a238573504868ba12dc5beb50f700/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:38.044853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91250-54f6-4977-9066-719215a30c3a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:38.045168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91250-54f6-4977-9066-719215a30c3a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:38.045465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf91250-54f6-4977-9066-719215a30c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:38.088548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-adf91250-54f6-4977-9066-719215a30c3a None None] MFA Rules not processed for user `b08a238573504868ba12dc5beb50f700`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:38.090696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-adf91250-54f6-4977-9066-719215a30c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:38.241568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 926/1811] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:38.256224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf91250-54f6-4977-9066-719215a30c3a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:38.258308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-adf91250-54f6-4977-9066-719215a30c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:38.280982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc24eb3d6174181b5e033872bf3a295'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:38.299353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc24eb3d6174181b5e033872bf3a295'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:38.374610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf91250-54f6-4977-9066-719215a30c3a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:38.379317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:38.379929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc24eb3d6174181b5e033872bf3a295'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:38.380412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:38.380777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc24eb3d6174181b5e033872bf3a295'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:38.381117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:38.381602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:38.381899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc24eb3d6174181b5e033872bf3a295'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:38.384460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:38.404424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 886/1812] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:37 2022] DELETE /identity/v3/users/0dc24eb3d6174181b5e033872bf3a295 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:38.479271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:38.481643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8f54a-92db-4b26-8246-3e89ea184a3a service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:38.511649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 927/1813] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:38.856532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:38.859049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:39.032420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770c4de-fbcb-425f-b7ad-a39480c158b6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:39.038385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:39.040140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.040140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.040825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] PATH_INFO: `/v3/projects/0b540ff750f442aa9de5211f4ab6f31d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.042407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b540ff750f442aa9de5211f4ab6f31d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:39.046895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:39.223198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:39.244073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b540ff750f442aa9de5211f4ab6f31d', 'request_id': 'req-b2ee2aec-280d-4487-8e83-55e8643d2946'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:39.351380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b540ff750f442aa9de5211f4ab6f31d', 'request_id': 'req-b2ee2aec-280d-4487-8e83-55e8643d2946'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:39.358603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b540ff750f442aa9de5211f4ab6f31d', 'request_id': 'req-b2ee2aec-280d-4487-8e83-55e8643d2946'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:39.375864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] The token cache is being invalidate because project 0b540ff750f442aa9de5211f4ab6f31d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:39.376304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:39.377062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b2ee2aec-280d-4487-8e83-55e8643d2946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:39.380490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 887/1814] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:38 2022] DELETE /identity/v3/projects/0b540ff750f442aa9de5211f4ab6f31d => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:39.385234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.386047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.386484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.417864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:39.419414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:39.441946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:39.495337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 928/1815] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:39.504435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.504798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.505092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.533230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:39.534497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:39.551665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 888/1816] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:39.552828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:39.566199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e005376-1a79-474c-a078-4aec83d6e8ed None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:39.571360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:39.572292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.572544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.573650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.573946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:39.574470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:39.585656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:39.606916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 929/1817] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:39.608266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:39.620972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f22342-8d39-4895-a866-bdc742c48fbe None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:39.634559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:39.635902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.636341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.636759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.639787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:39.642644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:39.788159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 889/1818] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:39 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:39.793065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:39.806179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6c060-0864-43d0-a76e-13a0c786597d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:39.917460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:39.917460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:39.917460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:39.917460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:39.917883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:39.917883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.045942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 930/1819] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:39 2022] POST /identity/v3/users => generated 284 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:40.049837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.078830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2724e0f-5228-4f57-aca9-ffb307e3fab7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.084642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.085557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.085922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.086276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.087392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.089168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.113690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 890/1820] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:40.117767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.138511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92706b0-9008-47bd-aed1-df786eaa95d8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.154376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.156837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.157148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.157526 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.158293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.161064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26170d1-fb02-4fe9-91a1-a4f0663b159e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.175895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 931/1821] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:29:40.180591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.201992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29362ca1-d3e2-4713-a644-ac88385d3cc8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.222975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.223897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.224184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.224467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] PATH_INFO: `/v3/projects/66f6562b1c684e94ba4af8072485631b/users/351ec08809ee42a9b94d3f90845dc46e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.246383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f6562b1c684e94ba4af8072485631b, user_id=351ec08809ee42a9b94d3f90845dc46e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.249431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.335725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 891/1822] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:40 2022] PUT /identity/v3/projects/66f6562b1c684e94ba4af8072485631b/users/351ec08809ee42a9b94d3f90845dc46e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:40.346780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.347337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.347626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.383123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] MFA Rules not processed for user `351ec08809ee42a9b94d3f90845dc46e`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:40.384368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:40.483356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 932/1823] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:40.488181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.556596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b900568-b20f-47b9-959c-5911b93cce37 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.562841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.563526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.563814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.564101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.564835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.566871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.660792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 892/1824] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:40 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:40.662008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.672802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f841ed-2f5f-4378-9138-576bfbf38fcc None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.691831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:40.737264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.743377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.743377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.743377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.743377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.743377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.886913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 933/1825] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:40 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:40.893266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.911470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0b52-9ff0-4f83-ae1c-71c9bb0af0c4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.917094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.917893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.918339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.919403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.920260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.922190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:40.940560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 893/1826] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:40.942194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:40.963523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033df6-fc41-44fd-947f-6076026cd49e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:40.970528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:40.970528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:40.970528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:40.970528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] PATH_INFO: `/v3/projects/2615a9878a66433e8622efc88fa2099c/users/14001e4fa02c46549a8090bd6787705d/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:40.988076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2615a9878a66433e8622efc88fa2099c, user_id=14001e4fa02c46549a8090bd6787705d, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:40.990972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:41.117127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 934/1827] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:40 2022] PUT /identity/v3/projects/2615a9878a66433e8622efc88fa2099c/users/14001e4fa02c46549a8090bd6787705d/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:41.125251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:41.136301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25938e-b142-4e41-aac5-0e280a3f82ad admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:41.160264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:41.217992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:41.218987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:41.219360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:41.220103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:41.220569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:41.222317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eecb7d-6cff-44d6-9abd-b79e3ffb4ca8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:41.237986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 894/1828] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:41.239185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:41.258603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822d12e-d7c3-4215-8faf-8ce1094cb4d8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:41.266870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:41.269211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:41.269736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:41.270389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] PATH_INFO: `/v3/projects/2615a9878a66433e8622efc88fa2099c/users/14001e4fa02c46549a8090bd6787705d/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:41.273333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2615a9878a66433e8622efc88fa2099c, user_id=14001e4fa02c46549a8090bd6787705d, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:41.285195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:41.300256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 935/1829] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:41 2022] PUT /identity/v3/projects/2615a9878a66433e8622efc88fa2099c/users/14001e4fa02c46549a8090bd6787705d/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:41.312333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:41.312741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:41.313036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:41.346612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] MFA Rules not processed for user `14001e4fa02c46549a8090bd6787705d`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:41.347785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:41.418412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 895/1830] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:41 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:41.430066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:41.433026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:41.497445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c9c6f-8cc4-44b9-a899-9d93147491be admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:41.502787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:41.503719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:41.504070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:41.504613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:41.516278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:41.517989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10f608-70a6-4ae1-9e81-df39d6887563 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:41.547924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 936/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:42.307460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:42.308898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:42.387727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed8e6-801b-460d-bc66-cb54fb8121a5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:42.392401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:42.393306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:42.393627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:42.393917 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] PATH_INFO: `/v3/users/607c44df3e30476d985e898762548d92` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:42.395220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607c44df3e30476d985e898762548d92)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:42.397253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:42.612273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607c44df3e30476d985e898762548d92'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:42.613510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:42.625785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607c44df3e30476d985e898762548d92'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:42.683128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607c44df3e30476d985e898762548d92'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:42.685102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607c44df3e30476d985e898762548d92'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:42.686867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607c44df3e30476d985e898762548d92'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:42.701331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 896/1832] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:42 2022] DELETE /identity/v3/users/607c44df3e30476d985e898762548d92 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:42.705966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6953-a754-448f-9835-1b818299291f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:42.706415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6953-a754-448f-9835-1b818299291f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:42.706723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6953-a754-448f-9835-1b818299291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:42.725075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-498f6953-a754-448f-9835-1b818299291f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:42.726412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-498f6953-a754-448f-9835-1b818299291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:42.797851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 937/1833] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:42.808463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:42.810542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:42.870689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7d445-d83e-4294-a1f7-3d7116fa08f0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:42.875174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:42.875813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:42.876072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:42.876348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:42.885896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:42.887454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:42.898463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 897/1834] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:43.359905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6953-a754-448f-9835-1b818299291f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:43.361681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-498f6953-a754-448f-9835-1b818299291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:43.378018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6953-a754-448f-9835-1b818299291f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:43.409884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:43.411102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:43.411389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:43.411668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] PATH_INFO: `/v3/projects/8cb36db936a740b7aa7c0e049917d608` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:43.413373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb36db936a740b7aa7c0e049917d608)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:43.415809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:43.436873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:43.592671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb36db936a740b7aa7c0e049917d608', 'request_id': 'req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.600574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb36db936a740b7aa7c0e049917d608', 'request_id': 'req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.677768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb36db936a740b7aa7c0e049917d608', 'request_id': 'req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.690522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] The token cache is being invalidate because project 8cb36db936a740b7aa7c0e049917d608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:43.690818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.691445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.693049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 938/1835] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:43 2022] DELETE /identity/v3/projects/8cb36db936a740b7aa7c0e049917d608 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:43.697138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:43.707365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836b61f-8a05-4e68-8c8b-b04c956620f7 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:43.770639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:43.771533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:43.771820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:43.772139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] PATH_INFO: `/v3/users/b08a238573504868ba12dc5beb50f700` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:43.773540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08a238573504868ba12dc5beb50f700)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:43.775487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:43.978253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08a238573504868ba12dc5beb50f700'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.993365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08a238573504868ba12dc5beb50f700'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:43.997835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:44.077745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08a238573504868ba12dc5beb50f700'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:44.078063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08a238573504868ba12dc5beb50f700'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:44.080055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08a238573504868ba12dc5beb50f700'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:44.093271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 898/1836] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:43 2022] DELETE /identity/v3/users/b08a238573504868ba12dc5beb50f700 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:44.508160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:44.509813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:44.512032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:44.514115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:44.525621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee81d75-3e9a-404e-a086-a73d6fe93fb5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:44.596879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:44.597174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6f82-ad49-4cd5-9cfc-0891639cdd9a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:44.597788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:44.598070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:44.598345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] PATH_INFO: `/v3/projects/4a48e0346ab344b7a3267458b812c5da` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:44.599503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a48e0346ab344b7a3267458b812c5da)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:44.601540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:44.601859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:44.602730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:44.603070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:44.603295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] PATH_INFO: `/v3/users/351ec08809ee42a9b94d3f90845dc46e` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:44.604488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351ec08809ee42a9b94d3f90845dc46e)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:44.606544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:45.316494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351ec08809ee42a9b94d3f90845dc46e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.324104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a48e0346ab344b7a3267458b812c5da', 'request_id': 'req-dd550263-7760-4d12-90e2-2e3874344d14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.331367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351ec08809ee42a9b94d3f90845dc46e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.335357 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a48e0346ab344b7a3267458b812c5da', 'request_id': 'req-dd550263-7760-4d12-90e2-2e3874344d14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.420549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351ec08809ee42a9b94d3f90845dc46e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.421496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351ec08809ee42a9b94d3f90845dc46e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.422112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-cf0d29ce-aa2a-452d-8cc3-ab0aaf9d8da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351ec08809ee42a9b94d3f90845dc46e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.436962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 899/1837] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:44 2022] DELETE /identity/v3/users/351ec08809ee42a9b94d3f90845dc46e => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:45.441189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.441608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.441905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.460331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:45.461570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:45.500190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a48e0346ab344b7a3267458b812c5da', 'request_id': 'req-dd550263-7760-4d12-90e2-2e3874344d14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.512069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] The token cache is being invalidate because project 4a48e0346ab344b7a3267458b812c5da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:45.513232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.513867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-dd550263-7760-4d12-90e2-2e3874344d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:45.515421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 939/1838] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:44 2022] DELETE /identity/v3/projects/4a48e0346ab344b7a3267458b812c5da => generated 0 bytes in 1008 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:45.520263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1508e-22a6-4756-a77c-928e186aaa30 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.520742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1508e-22a6-4756-a77c-928e186aaa30 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.521031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1508e-22a6-4756-a77c-928e186aaa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.523115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 900/1839] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:45.532578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:45.533968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:45.539154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d3d1508e-22a6-4756-a77c-928e186aaa30 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:45.540328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d3d1508e-22a6-4756-a77c-928e186aaa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:45.543226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:45.548433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 940/1840] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:45.556065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.556414 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.556693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.576439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:45.577710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:45.587143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 941/1841] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:45.612478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:45.612478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563750b-bd40-4e37-9c76-32c0ef5121ad None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:45.612801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10105872-f4ad-4765-a8ff-2d84b61b0df9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:45.612801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:45.613011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:45.613131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.613255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.613366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.613366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.613534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.613534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.613534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:45.613534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:45.613534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:45.617493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:45.618089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:45.621009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 942/1842] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:45.626294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:45.629934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 901/1843] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:29:45.631628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:45.636640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8aabb-87cf-47fa-a7c7-84b6b4f73c86 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:45.642694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:45.648005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.648005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.648005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.648005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:45.648005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:45.736129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 943/1844] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:45 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:45.738733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:45.764708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf0f6-c397-4f13-823a-a499051882d7 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:45.837528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:45.838248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:45.838530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:45.838806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:45.839496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:45.841493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.011120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 902/1845] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:45 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:46.011217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.020782 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f4da4-7462-49fa-b61a-70bb6034e265 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.025547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.026140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.026429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.026746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.027439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.029246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.057389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 944/1846] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:46.061880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.075899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2f62f-ca0c-497d-83c0-2fa32a8915e5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.082241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.083387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.083857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.084335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.085253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.087090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa051c-898f-45ea-bd23-0e52783bc4d5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.104714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.114804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 903/1847] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:46.117087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae97000-2bed-443a-ab85-0de74eb04a6e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.122242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.123854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.124331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.124690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] PATH_INFO: `/v3/projects/3bef23c708304098b2e6912aae0ef364/users/ac601c4fb5784ff4ace1685ba7283f8f/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.145614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bef23c708304098b2e6912aae0ef364, user_id=ac601c4fb5784ff4ace1685ba7283f8f, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.148619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.240840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 945/1848] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:46 2022] PUT /identity/v3/projects/3bef23c708304098b2e6912aae0ef364/users/ac601c4fb5784ff4ace1685ba7283f8f/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:46.253255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080323-a769-483a-982c-a74066eea7ba None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.253716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080323-a769-483a-982c-a74066eea7ba None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.254069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080323-a769-483a-982c-a74066eea7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.297553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-32080323-a769-483a-982c-a74066eea7ba None None] MFA Rules not processed for user `ac601c4fb5784ff4ace1685ba7283f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:46.302867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-32080323-a769-483a-982c-a74066eea7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:46.367059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 904/1849] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:46.378213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.380330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:46.392871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080323-a769-483a-982c-a74066eea7ba None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.394353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-32080323-a769-483a-982c-a74066eea7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:46.423589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:46.458549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f557ef-d93c-42e9-aae0-333075417d6d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.464142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.464898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.465278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.467425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.478976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.480700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.489226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080323-a769-483a-982c-a74066eea7ba None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.496371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.496849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 946/1850] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:46.498656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.498960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.499336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] PATH_INFO: `/v3/projects/66f6562b1c684e94ba4af8072485631b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.500578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f6562b1c684e94ba4af8072485631b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.503649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:46.662604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f6562b1c684e94ba4af8072485631b', 'request_id': 'req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:46.735447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f6562b1c684e94ba4af8072485631b', 'request_id': 'req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:46.742850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f6562b1c684e94ba4af8072485631b', 'request_id': 'req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:46.754908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] The token cache is being invalidate because project 66f6562b1c684e94ba4af8072485631b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:46.755241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:46.755929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:46.757586 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 905/1851] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:46 2022] DELETE /identity/v3/projects/66f6562b1c684e94ba4af8072485631b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:46.761680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:46.773395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee732a14-bbec-4b51-812e-a0ea54ea2765 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:46.834467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:46.835383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:46.835667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:46.835967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] PATH_INFO: `/v3/users/14001e4fa02c46549a8090bd6787705d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:46.837182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14001e4fa02c46549a8090bd6787705d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:46.839167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:47.096719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14001e4fa02c46549a8090bd6787705d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:47.097297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14001e4fa02c46549a8090bd6787705d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:47.099985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:47.163015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14001e4fa02c46549a8090bd6787705d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:47.163642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14001e4fa02c46549a8090bd6787705d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:47.176222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554d380b-9810-4503-8216-a200288a9357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14001e4fa02c46549a8090bd6787705d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:47.188719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 947/1852] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:46 2022] DELETE /identity/v3/users/14001e4fa02c46549a8090bd6787705d => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:47.853016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:47.854657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:47.869207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c60b8-b7c9-421d-96aa-24f4e3365a01 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:47.941961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:47.942929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:47.943350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:47.943735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] PATH_INFO: `/v3/projects/2615a9878a66433e8622efc88fa2099c` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:47.945061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2615a9878a66433e8622efc88fa2099c)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:47.948308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.052462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2615a9878a66433e8622efc88fa2099c', 'request_id': 'req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:48.054309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:48.163647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2615a9878a66433e8622efc88fa2099c', 'request_id': 'req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:48.170309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2615a9878a66433e8622efc88fa2099c', 'request_id': 'req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:48.182180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] The token cache is being invalidate because project 2615a9878a66433e8622efc88fa2099c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:48.182490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:48.183177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f9bdcdcb-0bce-4755-aa3b-e9d46f84ec69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:48.184845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 906/1853] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:47 2022] DELETE /identity/v3/projects/2615a9878a66433e8622efc88fa2099c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:48.191034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.191506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.191718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.211292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:48.212500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:48.274723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 948/1854] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:48.283061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21686c65-5e87-4302-8118-d882beae1678 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.283061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21686c65-5e87-4302-8118-d882beae1678 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.283061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21686c65-5e87-4302-8118-d882beae1678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.303596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-21686c65-5e87-4302-8118-d882beae1678 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:48.304833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-21686c65-5e87-4302-8118-d882beae1678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:48.314056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 907/1855] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:48.318103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.328164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9be85-1e8f-40fd-999a-ac3a89814c96 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.334924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.339502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.353442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 949/1856] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:48.359167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21686c65-5e87-4302-8118-d882beae1678 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.370276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21686c65-5e87-4302-8118-d882beae1678 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.376151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.376895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.377894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.377894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.378351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.380730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.469879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 908/1857] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:48 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:48.474319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.486299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998c0f-cb13-4321-8517-d0dc60bdfe47 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.544576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.544877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.547190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.547190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.547190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.548250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.655654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 950/1858] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:48 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:48.660483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.671324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b8815-1e09-4185-a529-cc7dcc1eeeb3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.676183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.676852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.677140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.677554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.678301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.679939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.696544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.697907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 909/1859] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:48.712594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4420455-3214-4394-a6ec-7450fe8249e5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.726167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.726167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.726167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.726167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.726755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.728510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b633f6-b220-48d5-ad7d-37d6b0f4d29f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.745611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 951/1860] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:48.746268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:48.760413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d70f6-e7c4-4b91-b901-ada257c89e83 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:48.765685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:48.766638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.767092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.767367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] PATH_INFO: `/v3/projects/0bdbb42c2cbd4615b96e805b9d02f67f/users/16294ca119a049379889bb9ae833cbe1/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.798713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdbb42c2cbd4615b96e805b9d02f67f, user_id=16294ca119a049379889bb9ae833cbe1, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:48.802164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:48.897113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 910/1861] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:48 2022] PUT /identity/v3/projects/0bdbb42c2cbd4615b96e805b9d02f67f/users/16294ca119a049379889bb9ae833cbe1/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:48.909535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:48.909836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:48.910124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:48.946826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] MFA Rules not processed for user `16294ca119a049379889bb9ae833cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:48.948318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:48.978317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:49.031778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 952/1862] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:49.038477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.117066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c457-b685-493d-8407-d6f897869ea8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.121942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.123310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.123597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.123988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.124692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.126661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.205895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 911/1863] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:49 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:49.209959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.221915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b98c74-7286-47ad-8a19-994b91da96aa None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.306671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.307650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.308044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.308447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.309366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.311784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.393673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 953/1864] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:49 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:49.397759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.409607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3a1fa-abbf-4f2e-9c33-a8d190d78b9c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.414404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.415082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.415371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.415655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.416385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.418038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.429759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 912/1865] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:49.433973 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.446658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7faaf-a089-41c4-8df7-935e229dd6a3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.452751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.453686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.453975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.454258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] PATH_INFO: `/v3/projects/b978b5167962421cae2116c1bb171a6b/users/ff0ec14051104ba4bfa6ebb7a54d3f4a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.471636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b978b5167962421cae2116c1bb171a6b, user_id=ff0ec14051104ba4bfa6ebb7a54d3f4a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.474546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.616691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 954/1866] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:49 2022] PUT /identity/v3/projects/b978b5167962421cae2116c1bb171a6b/users/ff0ec14051104ba4bfa6ebb7a54d3f4a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:49.622565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.633249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebb53-eb7d-4c34-a534-50482ca69599 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.697787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:49.715078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.715971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.716493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.716723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.717491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.719270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc53212-59f3-41d3-972c-a074236c2d32 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.730123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 913/1867] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:49.742074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.754619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40466c3f-31e6-4833-bf49-c8650d0e88ab admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:49.760024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:49.760918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.761203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.761515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] PATH_INFO: `/v3/projects/b978b5167962421cae2116c1bb171a6b/users/ff0ec14051104ba4bfa6ebb7a54d3f4a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.764454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b978b5167962421cae2116c1bb171a6b, user_id=ff0ec14051104ba4bfa6ebb7a54d3f4a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:49.768020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:49.784689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 955/1868] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:49 2022] PUT /identity/v3/projects/b978b5167962421cae2116c1bb171a6b/users/ff0ec14051104ba4bfa6ebb7a54d3f4a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:49.795823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:49.796008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:49.796942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:49.826319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] MFA Rules not processed for user `ff0ec14051104ba4bfa6ebb7a54d3f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:49.828656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:49.953273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 914/1869] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:49.970429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:49.973179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:50.069540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a1dab-fd25-4a02-a8c6-cd8969a5baa6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:50.073478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:50.074137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:50.074425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:50.074703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:50.085043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:50.086556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:50.101239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 956/1870] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:50.289621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:50.303791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff9973-15e1-4d18-9e77-4d2fc8fae74b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:50.304838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:50.306112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:50.306619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:50.306994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:50.368017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:50.370297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bf317-7f60-4290-84bc-a8e78207aa03 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:50.381105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 915/1871] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:50.621191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:50.622706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:50.645819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:50.694615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43548b0c-c1de-44ee-8860-b8445bd04d1c service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:50.699182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:50.700112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:50.700405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:50.700814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] PATH_INFO: `/v3/users/0c9fa62de02d4149aa0ceb4636628ea4` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:50.702048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9fa62de02d4149aa0ceb4636628ea4)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:50.703995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:51.063274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9fa62de02d4149aa0ceb4636628ea4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:51.063924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9fa62de02d4149aa0ceb4636628ea4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:51.180947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9fa62de02d4149aa0ceb4636628ea4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:51.181663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9fa62de02d4149aa0ceb4636628ea4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:51.195802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9fa62de02d4149aa0ceb4636628ea4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:51.210607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 957/1872] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:50 2022] DELETE /identity/v3/users/0c9fa62de02d4149aa0ceb4636628ea4 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:51.215425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:51.215843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:51.215843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:51.234566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:51.235787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:51.310612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 916/1873] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:51.315992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:51.317464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:51.388814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afca71-9b25-4790-9356-70d9ef21b0a2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:51.393436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:51.394214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:51.394565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:51.394914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:51.408917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:51.410330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:51.420529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 958/1874] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:51.839862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:51.841328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:51.853631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fbe24-a352-430a-8014-0888a76b08a8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:51.859719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:51.860622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:51.860906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:51.861183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] PATH_INFO: `/v3/projects/b9f649e42c7d41758c6c06a73435a96d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:51.863639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f649e42c7d41758c6c06a73435a96d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:51.865511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:51.868863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:51.966105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f649e42c7d41758c6c06a73435a96d', 'request_id': 'req-6d6f6b50-996c-42e6-bc64-0baa05effef0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.015596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f649e42c7d41758c6c06a73435a96d', 'request_id': 'req-6d6f6b50-996c-42e6-bc64-0baa05effef0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.021690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f649e42c7d41758c6c06a73435a96d', 'request_id': 'req-6d6f6b50-996c-42e6-bc64-0baa05effef0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.033373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] The token cache is being invalidate because project b9f649e42c7d41758c6c06a73435a96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:52.033686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.034346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.035871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 917/1875] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:51 2022] DELETE /identity/v3/projects/b9f649e42c7d41758c6c06a73435a96d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:52.039876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:52.050170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025d0-4fb2-4ccf-a1eb-3ae4a2c8ba7a service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:52.087527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:52.105020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:52.105921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:52.106201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:52.106475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] PATH_INFO: `/v3/users/38f057c4125a4f4ab14d506ead87fa1f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:52.108180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f057c4125a4f4ab14d506ead87fa1f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:52.110401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:52.295655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f057c4125a4f4ab14d506ead87fa1f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.296349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f057c4125a4f4ab14d506ead87fa1f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.398862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f057c4125a4f4ab14d506ead87fa1f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.399540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f057c4125a4f4ab14d506ead87fa1f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.418504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c2d32c6a-4525-4cfb-ba1c-a99c8723d658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f057c4125a4f4ab14d506ead87fa1f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:52.432635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 959/1876] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:52 2022] DELETE /identity/v3/users/38f057c4125a4f4ab14d506ead87fa1f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:52.971537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:52.973107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:52.986463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6b50-996c-42e6-bc64-0baa05effef0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:53.047117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:53.047915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.048197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.048479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] PATH_INFO: `/v3/projects/72a58ff568234bc5b242e68922945b3c` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.049691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a58ff568234bc5b242e68922945b3c)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:53.051606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:53.246041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a58ff568234bc5b242e68922945b3c', 'request_id': 'req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:53.387038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a58ff568234bc5b242e68922945b3c', 'request_id': 'req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:53.393869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a58ff568234bc5b242e68922945b3c', 'request_id': 'req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:53.405773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] The token cache is being invalidate because project 72a58ff568234bc5b242e68922945b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:53.406071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:53.406815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-fe9eb4b2-0f18-4622-a6c9-9d9292d70bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:53.408589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 918/1877] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:52 2022] DELETE /identity/v3/projects/72a58ff568234bc5b242e68922945b3c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:53.413425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.413900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.414017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.434632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:53.436791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:53.516636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 960/1878] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:53.526768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.527068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.527347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.546277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:53.547427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:53.555569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 919/1879] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:53.560014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:53.571059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087e8f3-1338-4b16-92c5-7abc72efb282 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:53.576833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:53.577532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.577873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.578107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.578848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:53.579158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:53.580778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:53.596582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 961/1880] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:29:53.597689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:53.607720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380132e4-9537-40f2-96f0-334b4cff4f9f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:53.612700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:53.613381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.613673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.613993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.614690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:53.616630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:53.631508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:53.731856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 920/1881] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:53 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:53.736945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:53.747304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08ed8-da02-4e64-9434-34ee71e7294c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:53.796673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:53.809437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:53.810603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:53.811057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:53.811342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:53.812110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:53.814072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:53.942780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 962/1882] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:53 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:53.954922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:53.975779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd1a1-4a0d-413a-b8ea-96f1933e9e51 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:53.999065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.000069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.000422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.000768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.001637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.003438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.016740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 921/1883] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:54.021361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.034450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e3d2-6d6e-43e1-a58b-a67417b1df94 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.043271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.044254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.044254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.045041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.045791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.047703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148a1ce-b4e2-4343-a3b1-8c078bf73c7d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.065135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 963/1884] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:54.069228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.079762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a9c9c-fac0-430b-913d-8f304874f050 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.085098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.086748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.087104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.087366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] PATH_INFO: `/v3/projects/0b272df8c1a84bf5b551f0867532ff28/users/3b560e8944554b1ba4a374552c00784a/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.104112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b272df8c1a84bf5b551f0867532ff28, user_id=3b560e8944554b1ba4a374552c00784a, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.106903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.181779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 922/1885] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:54 2022] PUT /identity/v3/projects/0b272df8c1a84bf5b551f0867532ff28/users/3b560e8944554b1ba4a374552c00784a/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:54.193448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.193728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.194034 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.220197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] MFA Rules not processed for user `3b560e8944554b1ba4a374552c00784a`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:54.223447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:54.287751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 964/1886] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:54.292284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.360449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3eb33-6212-44c3-96c1-d1ad84760b3f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.365281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.365974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.366262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.366544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.367252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.369176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.515987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 923/1887] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:54 2022] POST /identity/v3/projects => generated 389 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:54.519917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.532055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bf085-38ca-4ba0-b050-faaab2a1eb7e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.592359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.593035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.593323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.593629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.594330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.596295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.724983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 965/1888] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:54 2022] POST /identity/v3/users => generated 301 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:54.729012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.741074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79961176-1695-47e8-a6ea-46e320d7aa60 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.745883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.746751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.747038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.747314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.748263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.749888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.765125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 924/1889] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:29:54.772680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.789499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73d12f-12a7-406a-befd-1cd9f65c4f62 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.795809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.797033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.797843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.798290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] PATH_INFO: `/v3/projects/d81757e084a34395a98886c08965e538/users/ce64bdaa083a4bee9129c0ffc76ac3d6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.815309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81757e084a34395a98886c08965e538, user_id=ce64bdaa083a4bee9129c0ffc76ac3d6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.819058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:54.904144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 966/1890] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:54 2022] PUT /identity/v3/projects/d81757e084a34395a98886c08965e538/users/ce64bdaa083a4bee9129c0ffc76ac3d6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:54.908628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:54.919152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392611a-26fc-478b-8fd5-a329668e9284 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:54.991399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:54.992122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:54.994401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:54.994401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:54.994401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:54.995446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278bf5d8-8c23-44c7-ad62-1375a1b80597 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:55.007574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 925/1891] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:55.011422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:55.025669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61899234-f243-490e-8bfd-c21baad2b73f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:55.041888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:55.043136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:55.043136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:55.043413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] PATH_INFO: `/v3/projects/d81757e084a34395a98886c08965e538/users/ce64bdaa083a4bee9129c0ffc76ac3d6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:55.056364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81757e084a34395a98886c08965e538, user_id=ce64bdaa083a4bee9129c0ffc76ac3d6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:55.059343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:55.070567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 967/1892] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:55 2022] PUT /identity/v3/projects/d81757e084a34395a98886c08965e538/users/ce64bdaa083a4bee9129c0ffc76ac3d6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:55.081787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:55.082083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:55.082396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:55.110943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] MFA Rules not processed for user `ce64bdaa083a4bee9129c0ffc76ac3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:55.112255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:55.222481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 926/1893] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:55.234029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:55.236311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:55.308117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2b6cc-ac50-4b48-90b4-316de3554773 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:55.312884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:55.313728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:55.314088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:55.314442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:55.325832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:55.328410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:55.353638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 968/1894] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:55.364680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:55.479922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:55.494926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:55.558862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d9f51-42f4-4ff2-b025-10d8956c2890 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:55.570389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:55.570974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:55.571332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:55.571679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] PATH_INFO: `/v3/users/3b560e8944554b1ba4a374552c00784a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:55.573634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b560e8944554b1ba4a374552c00784a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:55.576611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:55.835173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b560e8944554b1ba4a374552c00784a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:55.846842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b560e8944554b1ba4a374552c00784a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:55.926899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:55.928760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:55.936856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b560e8944554b1ba4a374552c00784a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:55.944766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b560e8944554b1ba4a374552c00784a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:55.946119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ba838a81-843e-4c2e-b6ee-b10c17cefa30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b560e8944554b1ba4a374552c00784a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:55.965282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 927/1895] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:55 2022] DELETE /identity/v3/users/3b560e8944554b1ba4a374552c00784a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:55.968590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252aecc-aede-4a69-83ea-3db2f3d67d0e service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:55.970051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:55.970352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:55.970665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:55.989278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:55.992867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:56.038633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:56.039522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:56.039807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:56.040087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] PATH_INFO: `/v3/users/16294ca119a049379889bb9ae833cbe1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:56.041302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16294ca119a049379889bb9ae833cbe1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:56.043419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:56.056838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 928/1896] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:56.066796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:56.068342 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:56.132915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c81991-4542-4b6a-843b-fb0d9e75e8bc None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:56.137287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:56.138067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:56.138432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:56.139026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:56.150610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:56.156070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:56.156070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb37665-7b18-4e4f-ae42-830c0f9122d5 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:56.167973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 929/1897] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:56.260895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16294ca119a049379889bb9ae833cbe1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.261788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16294ca119a049379889bb9ae833cbe1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.353653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16294ca119a049379889bb9ae833cbe1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.354453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16294ca119a049379889bb9ae833cbe1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.369129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16294ca119a049379889bb9ae833cbe1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.384564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 969/1898] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:55 2022] DELETE /identity/v3/users/16294ca119a049379889bb9ae833cbe1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:56.390067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:56.390569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:56.391098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:56.416642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:56.417929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:56.492351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 930/1899] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:56.502005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:56.503427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:56.538038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:56.566002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d50d5e-871d-4f46-b0ab-89dc664d6a41 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:56.569638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:56.570290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:56.570578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:56.570860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:56.581804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:56.583343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:56.596436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 970/1900] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:29:56.672082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:56.673995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:56.692163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bfa95-40d3-41a1-9633-8bb336ece525 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:56.698019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:56.698770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:56.706767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:56.707066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] PATH_INFO: `/v3/projects/0b272df8c1a84bf5b551f0867532ff28` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:56.708386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b272df8c1a84bf5b551f0867532ff28)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:56.710771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:56.874842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b272df8c1a84bf5b551f0867532ff28', 'request_id': 'req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.970371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b272df8c1a84bf5b551f0867532ff28', 'request_id': 'req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.976634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b272df8c1a84bf5b551f0867532ff28', 'request_id': 'req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.986186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:56.990054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] The token cache is being invalidate because project 0b272df8c1a84bf5b551f0867532ff28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:56.990347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.991004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:56.992673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 931/1901] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:56 2022] DELETE /identity/v3/projects/0b272df8c1a84bf5b551f0867532ff28 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:56.996653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:57.006760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb0dbf-1b6f-40f2-a6c7-6d39fd2a585f service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:57.062972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:57.063834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:57.064319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:57.064439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:57.064875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] PATH_INFO: `/v3/users/ce64bdaa083a4bee9129c0ffc76ac3d6` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:57.066069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:57.067896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce64bdaa083a4bee9129c0ffc76ac3d6)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:57.069808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:57.082258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebd18-4b73-4379-82df-1d6e6d17d5d8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:57.087712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:57.088684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:57.088971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:57.089251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] PATH_INFO: `/v3/projects/0bdbb42c2cbd4615b96e805b9d02f67f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:57.090562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdbb42c2cbd4615b96e805b9d02f67f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:57.093264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:57.388427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce64bdaa083a4bee9129c0ffc76ac3d6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.389326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce64bdaa083a4bee9129c0ffc76ac3d6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.390974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdbb42c2cbd4615b96e805b9d02f67f', 'request_id': 'req-003091f4-49ce-4667-8bbf-afbc7d182126'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.515000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdbb42c2cbd4615b96e805b9d02f67f', 'request_id': 'req-003091f4-49ce-4667-8bbf-afbc7d182126'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.523065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdbb42c2cbd4615b96e805b9d02f67f', 'request_id': 'req-003091f4-49ce-4667-8bbf-afbc7d182126'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.539907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] The token cache is being invalidate because project 0bdbb42c2cbd4615b96e805b9d02f67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:57.540695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.547843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.550557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 932/1902] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:57 2022] DELETE /identity/v3/projects/0bdbb42c2cbd4615b96e805b9d02f67f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:57.554059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:57.556297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:57.610933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce64bdaa083a4bee9129c0ffc76ac3d6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.611600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce64bdaa083a4bee9129c0ffc76ac3d6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.636009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce64bdaa083a4bee9129c0ffc76ac3d6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:57.678138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 971/1903] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:56 2022] DELETE /identity/v3/users/ce64bdaa083a4bee9129c0ffc76ac3d6 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:57.694164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003091f4-49ce-4667-8bbf-afbc7d182126 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:57.766608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:57.767487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:57.767767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:57.768049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] PATH_INFO: `/v3/users/ff0ec14051104ba4bfa6ebb7a54d3f4a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:57.769359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0ec14051104ba4bfa6ebb7a54d3f4a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:57.771296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:58.241768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:58.256258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0ec14051104ba4bfa6ebb7a54d3f4a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.269629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0ec14051104ba4bfa6ebb7a54d3f4a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.290393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:58.292320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:58.308085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae727c6d-53b7-4219-a732-11cc7c4e8897 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:58.316312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:58.316312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:58.316312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:58.316312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] PATH_INFO: `/v3/projects/d81757e084a34395a98886c08965e538` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:58.316937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81757e084a34395a98886c08965e538)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:58.319015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:58.399368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0ec14051104ba4bfa6ebb7a54d3f4a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.400022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0ec14051104ba4bfa6ebb7a54d3f4a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.400655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c48c0008-17bb-4599-b853-dc5350aa1b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0ec14051104ba4bfa6ebb7a54d3f4a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.413720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 933/1904] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:29:57 2022] DELETE /identity/v3/users/ff0ec14051104ba4bfa6ebb7a54d3f4a => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:58.448378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:58.448604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81757e084a34395a98886c08965e538', 'request_id': 'req-cace2c7e-fa3f-44a3-bb33-5175eb1df832'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.455334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81757e084a34395a98886c08965e538', 'request_id': 'req-cace2c7e-fa3f-44a3-bb33-5175eb1df832'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.536161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81757e084a34395a98886c08965e538', 'request_id': 'req-cace2c7e-fa3f-44a3-bb33-5175eb1df832'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.549168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] The token cache is being invalidate because project d81757e084a34395a98886c08965e538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:58.549594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.550241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-cace2c7e-fa3f-44a3-bb33-5175eb1df832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:58.552111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 972/1905] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:58 2022] DELETE /identity/v3/projects/d81757e084a34395a98886c08965e538 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:29:58.557633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:58.557927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:58.558240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:58.577411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:58.580131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:58.644626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 934/1906] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:58.651560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:58.652492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:58.652791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:58.672760 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:58.674008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:58.689982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 973/1907] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:58.696206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:58.717369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d007fc-a0b2-4e5a-b1fd-304293e3858f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:58.725668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:58.726682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:58.727071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:58.727456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:58.728385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:58.728994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:58.731224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:58.752573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 935/1908] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:29:58.757605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:58.768446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec285-6cb3-4a59-828e-6c7bc4252859 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:58.773660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:58.781186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:58.781186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:58.781186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:58.781186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:58.781186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:58.988194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 974/1909] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:58 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:58.992247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.003564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e4ed-009c-44f2-83d3-cffd7d51e7ba admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.063366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.064139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.064422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.064720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.065454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.078745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.183758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.184285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 936/1910] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:58 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:29:59.186162 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:59.188882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.203524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f0caa-4171-4717-8982-9717238b517c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.205380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5436f-2dc6-4bda-aa75-73dfa03fb36b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.209712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.210399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.210688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.210795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.210988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.211867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.212193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.212533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.212709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] PATH_INFO: `/v3/projects/b978b5167962421cae2116c1bb171a6b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.213912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b978b5167962421cae2116c1bb171a6b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.214815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.216062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.228334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 937/1911] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:59.232538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.244168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac62b8-7c9a-43ec-9e4b-1ee2a4f2e09d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.249375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.250047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.250332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.257387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.257387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.257387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.264237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 938/1912] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:29:59.268756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.278430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26839c7-5cd5-44a5-822a-1d12f19d828f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.283631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.284495 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.284789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.285053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] PATH_INFO: `/v3/projects/0f882b896a8a4db4acf8b8cf1ed03029/users/deb001871ec54a1bbbc3cbf067c057fa/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.300385 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f882b896a8a4db4acf8b8cf1ed03029, user_id=deb001871ec54a1bbbc3cbf067c057fa, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.303138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bea44-ca0a-4be4-a987-17ddd4ed2f7e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.401476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 939/1913] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:29:59 2022] PUT /identity/v3/projects/0f882b896a8a4db4acf8b8cf1ed03029/users/deb001871ec54a1bbbc3cbf067c057fa/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:59.418689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b978b5167962421cae2116c1bb171a6b', 'request_id': 'req-54741c12-4009-4fd1-a5b5-d7b7bc05d865'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:59.422330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.422795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.423499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.427572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b978b5167962421cae2116c1bb171a6b', 'request_id': 'req-54741c12-4009-4fd1-a5b5-d7b7bc05d865'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:59.459068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] MFA Rules not processed for user `deb001871ec54a1bbbc3cbf067c057fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:59.460463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:59.519420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b978b5167962421cae2116c1bb171a6b', 'request_id': 'req-54741c12-4009-4fd1-a5b5-d7b7bc05d865'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:59.532850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 940/1914] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:59.536843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] The token cache is being invalidate because project b978b5167962421cae2116c1bb171a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:29:59.538556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:59.538662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.538774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-54741c12-4009-4fd1-a5b5-d7b7bc05d865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:29:59.539593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 975/1915] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:29:59 2022] DELETE /identity/v3/projects/b978b5167962421cae2116c1bb171a6b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:29:59.544251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ce9-39f1-42e6-b594-546763003b6b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.544544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ce9-39f1-42e6-b594-546763003b6b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.544826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01ce9-39f1-42e6-b594-546763003b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.563102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2ea01ce9-39f1-42e6-b594-546763003b6b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:59.564365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2ea01ce9-39f1-42e6-b594-546763003b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:59.606741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb81813-617a-4049-b7ab-87f8279ce6de None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.611951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.612610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.612892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.613170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.613887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.615815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.626963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 976/1916] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:29:59.634774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.635069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.635360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.651311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:29:59.657244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:29:59.658561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:59.671795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 977/1917] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:29:59.677739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.687903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1776c-f82c-4c52-9753-2e2926ffc523 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.693074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.693766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.694048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.694321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.695137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:29:59.695438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.697050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.711089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 978/1918] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:29:59.715942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.726633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298fe0c-22a5-4214-89b9-e2d9b916a79b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.731463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.732125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.732412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.732745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.733631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.735810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.760988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 941/1919] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/projects => generated 367 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:59.765839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.767612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:29:59.832895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3118be-c2da-4065-8170-e01e632a3a6d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.837670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.838371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.838659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.838942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.839640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.841855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.863793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 979/1920] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:59.867974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.928938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0c487-4d60-4368-95d3-b69465a1294b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:29:59.933649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:29:59.934323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:29:59.934612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:29:59.934895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:29:59.935584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:29:59.938302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:29:59.982679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 942/1921] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:29:59.987421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:29:59.997617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0050c43-2dc4-4a27-93e6-33939788794b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.003309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.003979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.004267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.004546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.005274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.006938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.010811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:00.021778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 943/1922] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:00.023169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.037541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e334e01-8a87-464e-9a62-74cd2a91db2e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.042312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.043230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.043488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.043765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] PATH_INFO: `/v3/projects/45e26d60656942d78e9d6a6eb932f190/users/e014f77f3c7647b188f015fffe90da8a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.065641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 980/1923] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:29:59 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:00.067983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e26d60656942d78e9d6a6eb932f190, user_id=e014f77f3c7647b188f015fffe90da8a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.070118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.073023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.084064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422288-8e2b-45d2-b347-f84be3ffd5b7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.090746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.091419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.091705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.092017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.092900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.094680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.105305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 981/1924] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:00.110200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.120634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51fc8b-91af-4d21-822d-bbe65a5b2a63 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.128371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.131499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.131499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.131499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.131499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.132558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.143904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 982/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:00.148616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.156755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 944/1926] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:00 2022] PUT /identity/v3/projects/45e26d60656942d78e9d6a6eb932f190/users/e014f77f3c7647b188f015fffe90da8a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:00.160686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.167963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0a91c-36f6-41f6-a3ae-ddc6c812e609 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.243246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.244523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.244966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.245648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] PATH_INFO: `/v3/projects/90a92435575244d8af279d841029e198/users/e8a269cc0da144e59cafdc70ac241b2a/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.248985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c4b3-3fa3-47e6-8bf6-6134df2227ae admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.255629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.256375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.256749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.257181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.258020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.259629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.273380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 945/1927] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:00.273578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a92435575244d8af279d841029e198, user_id=e8a269cc0da144e59cafdc70ac241b2a, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.273867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8942f-e02d-4495-b2db-d60f4f707f1a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.275100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.285916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457e82-b11e-4ce8-a1cc-40e30f0bf46b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.290884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.291792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.292079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.292367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] PATH_INFO: `/v3/projects/45e26d60656942d78e9d6a6eb932f190/users/e014f77f3c7647b188f015fffe90da8a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.294393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e26d60656942d78e9d6a6eb932f190, user_id=e014f77f3c7647b188f015fffe90da8a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.297836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfaf77-6fb5-4234-9ff4-de5df4f89f00 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.308732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 946/1928] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:00 2022] PUT /identity/v3/projects/45e26d60656942d78e9d6a6eb932f190/users/e014f77f3c7647b188f015fffe90da8a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:00.325515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.325820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.326212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.356671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] MFA Rules not processed for user `e014f77f3c7647b188f015fffe90da8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:00.358839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:00.412113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 983/1929] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:00 2022] PUT /identity/v3/projects/90a92435575244d8af279d841029e198/users/e8a269cc0da144e59cafdc70ac241b2a/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:00.424111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.424714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.424714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.434649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 947/1930] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:00.447397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.448645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:00.460187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] MFA Rules not processed for user `e8a269cc0da144e59cafdc70ac241b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:00.461660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:00.520618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab75d9-2a09-47c9-a7b9-197d8013d154 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.541742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 984/1931] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:00.546547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.573963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.574634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.574924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.575203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.626347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53f01c-368e-4095-99fa-5e2a9b4d0ed6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.631378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.632132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.632558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.632859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.633639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.635572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.655606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.657254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.667490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 948/1932] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:00.731260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:00.738634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 985/1933] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:00 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:00.743167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.754593 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656585c-abc7-419f-910e-7de83ebe9541 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.813821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.814570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.814968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.815286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.816124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.818257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.834093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:00.908696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 949/1934] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:00 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:00.912862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.924324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf7b36-bad9-4cdf-baf9-e52d979d6de0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.929701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.930382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.930672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.930953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.931690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.933320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.945257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 986/1935] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:00.949999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.961006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abaf20a-70f5-4d0e-9056-98bbc428cde4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:00.966064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.968542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:00.968542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:00.968542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:00.968542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:00.968877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:00.970480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:00.980915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 950/1936] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:00.986013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:00.995786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8b45-ec88-4b19-9356-3c1ef02d3b57 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.002053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.002935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.003262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.003543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] PATH_INFO: `/v3/projects/2752f65b5af5411cb7978f470d14ac42/users/0d2d8b862c7f4be6bad0452c62f2d707/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.018676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2752f65b5af5411cb7978f470d14ac42, user_id=0d2d8b862c7f4be6bad0452c62f2d707, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.021703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd22ff-6b2e-40ae-b321-08a1f1ab686d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.023677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3af2-e1db-4ee7-9450-738460bc6fbf admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.027544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.028197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.028486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.028767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.030622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:01.074069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:01.091936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.093656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.102162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 951/1937] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:00 2022] PUT /identity/v3/projects/2752f65b5af5411cb7978f470d14ac42/users/0d2d8b862c7f4be6bad0452c62f2d707/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:01.105603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 987/1938] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:01.113219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.113426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.113711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.126636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:01.141987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] MFA Rules not processed for user `0d2d8b862c7f4be6bad0452c62f2d707`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:01.143388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:01.224375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 952/1939] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:01.229001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:01.240747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b155c4-e74d-4bcf-af18-79db032c337a service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.307817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.310542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.310542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.310542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.310542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.312886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.360736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:01.475234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 988/1940] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:01 2022] POST /identity/v3/projects => generated 377 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:01.479804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:01.494675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26504b64-0d4d-4884-922e-843aa60a7bff None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.555459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.556164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.556473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.556749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.557480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.559434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.663439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 953/1941] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:01 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:01.667614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:01.681957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca36df-7372-4e3b-8d9d-1c8acea8b428 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.689123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.690139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.690383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.690684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.691415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.693035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.707553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 989/1942] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:01.708778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:01.719642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ef0d7-c6b5-4477-9615-e349cbcf7f9d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.724374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.725259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.725589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.725877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] PATH_INFO: `/v3/projects/0a098ad584924616819dfe1cc43af023/users/4c56d2fb57a04e69a5e5a672e55f47cb/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.742166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a098ad584924616819dfe1cc43af023, user_id=4c56d2fb57a04e69a5e5a672e55f47cb, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.744884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.856789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 954/1943] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:01 2022] PUT /identity/v3/projects/0a098ad584924616819dfe1cc43af023/users/4c56d2fb57a04e69a5e5a672e55f47cb/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:01.861691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:01.872250 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005298-a897-4936-a942-8c8525518607 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:01.970804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:01.981157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:01.981157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:01.981157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:01.981157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:01.981157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79816cd-8971-4da0-8729-6cbbc9568ad1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:01.985565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 990/1944] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:01.989525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:02.000168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1e485-86f5-431a-8205-d689c6f3f74b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:02.004693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:02.005598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:02.005881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:02.006159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] PATH_INFO: `/v3/projects/0a098ad584924616819dfe1cc43af023/users/4c56d2fb57a04e69a5e5a672e55f47cb/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:02.008772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a098ad584924616819dfe1cc43af023, user_id=4c56d2fb57a04e69a5e5a672e55f47cb, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:02.011487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:02.022681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 955/1945] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:01 2022] PUT /identity/v3/projects/0a098ad584924616819dfe1cc43af023/users/4c56d2fb57a04e69a5e5a672e55f47cb/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:02.032945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:02.033367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:02.033669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:02.060597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] MFA Rules not processed for user `4c56d2fb57a04e69a5e5a672e55f47cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:02.062507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:02.113372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:02.132226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 991/1946] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:02.145092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:02.169320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:02.209476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d43f6-0d85-467f-98bc-da6515d59b8c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:02.211684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:02.212265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:02.212555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:02.212852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:02.228330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:02.228330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:02.239963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 956/1947] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:02.398620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:02.407925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c21f30-1903-45b1-a122-f6411094fb8f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:02.411374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:02.412010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:02.412291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:02.412566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:02.471601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:02.473119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79f096-e70b-4eb5-9c98-25725e875ba5 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:02.483477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 992/1948] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:10.410212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:10.520945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e76a4-33b0-4a23-a442-7a09ac38fdf4 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:10.525884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:10.526777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:10.527058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:10.527329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] PATH_INFO: `/v3/users/e8a269cc0da144e59cafdc70ac241b2a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:10.528557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a269cc0da144e59cafdc70ac241b2a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:10.530573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:10.567550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b0c1d-d59b-42cf-a767-2afac76f4566 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:10.568078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b0c1d-d59b-42cf-a767-2afac76f4566 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:10.568078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b0c1d-d59b-42cf-a767-2afac76f4566 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:10.569481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 993/1949] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 16:30:10.579078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:10.582000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:10.582000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:10.610341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:10.612207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:10.695669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 994/1950] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:30:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:10.706568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:10.707913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:10.720717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc0a2-f3d2-42c8-90d5-55f5bee91811 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:10.725846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:10.726479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:10.726764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:10.727045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:10.737364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:10.738989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:10.751367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 995/1951] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:10.790874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:10.792524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:10.801791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a269cc0da144e59cafdc70ac241b2a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:10.819580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a269cc0da144e59cafdc70ac241b2a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:10.826144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d771b9b-01dc-4e4a-9178-385205dbb9b9 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:10.834617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:10.835629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:10.835920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:10.836500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] PATH_INFO: `/v3/users/a0a422ada9ab43609a7940a1df349b7b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:10.837796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a422ada9ab43609a7940a1df349b7b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:10.840127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:11.023693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a269cc0da144e59cafdc70ac241b2a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.024409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a269cc0da144e59cafdc70ac241b2a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.025104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-316df4ae-4f85-4374-8fb5-ec95a80ba20e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a269cc0da144e59cafdc70ac241b2a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.035066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:11.039597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 957/1952] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:10 2022] DELETE /identity/v3/users/e8a269cc0da144e59cafdc70ac241b2a => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:11.044094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.044383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.044656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.064303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:11.100650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:11.101396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a422ada9ab43609a7940a1df349b7b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.102241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a422ada9ab43609a7940a1df349b7b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.105915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.173311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 958/1953] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:11.183215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:11.183799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a422ada9ab43609a7940a1df349b7b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.184484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a422ada9ab43609a7940a1df349b7b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.184810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.198104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a422ada9ab43609a7940a1df349b7b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.211853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 996/1954] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:10 2022] DELETE /identity/v3/users/a0a422ada9ab43609a7940a1df349b7b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:11.237083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:11.239675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.279722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cff71-1b4c-43e3-8722-506227fcfc45 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:11.319485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b00b4-9a7c-423b-b64b-1a71682b2a21 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:11.326107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:11.326107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.326107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.326107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] PATH_INFO: `/v3/users/deb001871ec54a1bbbc3cbf067c057fa` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.327009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb001871ec54a1bbbc3cbf067c057fa)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:11.329574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:11.339941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:11.340598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.340883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.341153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.356334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:11.357850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:11.369912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 959/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:11.562805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:11.564508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.578012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2328029-a145-4d1b-aacd-233fb6d82777 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:11.582548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:11.584086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.584377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.584655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] PATH_INFO: `/v3/projects/32666b3567c94e06a7f720c1abe2a8b7` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.585915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] RBAC: Authorizing `identity:delete_project(project_id=32666b3567c94e06a7f720c1abe2a8b7)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:11.587854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:11.607065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb001871ec54a1bbbc3cbf067c057fa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.608167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb001871ec54a1bbbc3cbf067c057fa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.708589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb001871ec54a1bbbc3cbf067c057fa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.709378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb001871ec54a1bbbc3cbf067c057fa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.727454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ceff51b8-9d8b-417e-8380-57f6240ef383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb001871ec54a1bbbc3cbf067c057fa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.732195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32666b3567c94e06a7f720c1abe2a8b7', 'request_id': 'req-824febc8-b8ad-4b24-b61e-81f50b26569b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.742748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 997/1956] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:11 2022] DELETE /identity/v3/users/deb001871ec54a1bbbc3cbf067c057fa => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:11.747295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556da77-8bd8-40a0-8bff-cc1ff7ac3c0d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.747754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556da77-8bd8-40a0-8bff-cc1ff7ac3c0d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.748043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556da77-8bd8-40a0-8bff-cc1ff7ac3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.766513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3556da77-8bd8-40a0-8bff-cc1ff7ac3c0d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:11.767786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3556da77-8bd8-40a0-8bff-cc1ff7ac3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.792097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:11.817288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32666b3567c94e06a7f720c1abe2a8b7', 'request_id': 'req-824febc8-b8ad-4b24-b61e-81f50b26569b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.823821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32666b3567c94e06a7f720c1abe2a8b7', 'request_id': 'req-824febc8-b8ad-4b24-b61e-81f50b26569b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.829663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 998/1957] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:11.835748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] The token cache is being invalidate because project 32666b3567c94e06a7f720c1abe2a8b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:11.836047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.837181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:11.841773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 960/1958] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:11 2022] DELETE /identity/v3/projects/32666b3567c94e06a7f720c1abe2a8b7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:11.846954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.847447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.847549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.848401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:11.849881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.901547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:11.902902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.913795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 999/1959] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:11.914836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:11.917051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:11.928830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824febc8-b8ad-4b24-b61e-81f50b26569b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:11.929784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f063ba-8ccb-4881-9039-bd052f7dc8f8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:11.939058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:11.940724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:11.941086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:11.941483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] PATH_INFO: `/v3/projects/90a92435575244d8af279d841029e198` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:11.948282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a92435575244d8af279d841029e198)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:11.953553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:11.973084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:12.002773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.003433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.003717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.003997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.005983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.018631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.020153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4aaac-987b-43d8-ac24-e1b06de4c3e2 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.029899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 961/1960] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:12.031918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.032330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.032765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.052340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:12.053796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.063568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 962/1961] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:12.068193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.078959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58575311-b919-49b4-9bea-cfdec2d74665 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.084002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a92435575244d8af279d841029e198', 'request_id': 'req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.089752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a92435575244d8af279d841029e198', 'request_id': 'req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.143578 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.144305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.144611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.144920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.145742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:12.146108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.148004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.160938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 963/1962] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:30:12.165010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.173032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a92435575244d8af279d841029e198', 'request_id': 'req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.174867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec64ee5-eeb9-4df1-bcf5-1806f11142dd admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.179808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.180455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.182944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.182944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.182944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.183665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.186969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] The token cache is being invalidate because project 90a92435575244d8af279d841029e198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:12.186969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.186969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.188969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1000/1963] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:11 2022] DELETE /identity/v3/projects/90a92435575244d8af279d841029e198 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:12.192991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.194046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.227304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ab01f-0bd9-4dde-9fcd-7054e77e6cb9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.227304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.227304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.227304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.227691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] PATH_INFO: `/v3/users/0d2d8b862c7f4be6bad0452c62f2d707` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.228865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2d8b862c7f4be6bad0452c62f2d707)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.231045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.291704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 964/1964] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:12 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:12.317474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.319492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.405147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476bc0-89e2-4664-960c-6b7faec61695 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.410548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.411224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.411510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.411788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.412482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.414494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.429793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:12.506344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2d8b862c7f4be6bad0452c62f2d707'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.507082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2d8b862c7f4be6bad0452c62f2d707'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.547047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 965/1965] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:12 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:12.547047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.547047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.561269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017ec6f-fe99-4334-b27f-567bb01ee41e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.567431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.570620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.570620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.570620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] PATH_INFO: `/v3/projects/0f882b896a8a4db4acf8b8cf1ed03029` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.570620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f882b896a8a4db4acf8b8cf1ed03029)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.572274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.628887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2d8b862c7f4be6bad0452c62f2d707'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.629560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2d8b862c7f4be6bad0452c62f2d707'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.642556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2d8b862c7f4be6bad0452c62f2d707'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.654518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1001/1966] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:12 2022] DELETE /identity/v3/users/0d2d8b862c7f4be6bad0452c62f2d707 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:12.657625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.657625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.662060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:12.714783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f882b896a8a4db4acf8b8cf1ed03029', 'request_id': 'req-df55fe95-4b34-4736-9062-c61d14e65c33'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.729501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1eabbd-77c7-4fd2-9657-c1a3c88ab4ec admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.788806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.789518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.789809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.790091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.790823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.792445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.804623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1002/1967] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:12.808613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.819513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fd85-1848-402c-a296-c3607cbe43dc admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.824617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.825271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.825601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.825883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.826609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.828322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.838362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1003/1968] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:12.840577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f882b896a8a4db4acf8b8cf1ed03029', 'request_id': 'req-df55fe95-4b34-4736-9062-c61d14e65c33'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.843805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.848592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f882b896a8a4db4acf8b8cf1ed03029', 'request_id': 'req-df55fe95-4b34-4736-9062-c61d14e65c33'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.856724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c08c69-bd56-4206-ab96-ce4ca9d7c2bd admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.863008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] The token cache is being invalidate because project 0f882b896a8a4db4acf8b8cf1ed03029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:12.863209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.863324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.863683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.863979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.864087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:12.864762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] PATH_INFO: `/v3/projects/cd7f849538d44293999c28b49a00bed8/users/54cfca53919a42b2a420512b60d43b36/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.866331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 966/1969] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:12 2022] DELETE /identity/v3/projects/0f882b896a8a4db4acf8b8cf1ed03029 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:12.870265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:12.871967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:12.881943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7f849538d44293999c28b49a00bed8, user_id=54cfca53919a42b2a420512b60d43b36, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.885365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84a6c4-b12f-4478-a489-ca8829e5355c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:12.894346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55fe95-4b34-4736-9062-c61d14e65c33 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:12.902754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:12.904872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:12.905468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:12.906099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] PATH_INFO: `/v3/users/e014f77f3c7647b188f015fffe90da8a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:12.907779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e014f77f3c7647b188f015fffe90da8a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:12.910417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:13.002782 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1004/1970] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:12 2022] PUT /identity/v3/projects/cd7f849538d44293999c28b49a00bed8/users/54cfca53919a42b2a420512b60d43b36/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:13.013907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:13.014298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:13.014655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:13.041292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] MFA Rules not processed for user `54cfca53919a42b2a420512b60d43b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:13.042546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:13.105820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1005/1971] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:13.132623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:13.134035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:13.210079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7241aa-20eb-4b5f-8c6c-04d8c89d377e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:13.210983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e014f77f3c7647b188f015fffe90da8a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.212465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:13.216966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:13.217664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:13.217955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:13.218907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:13.227852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:13.229459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:13.233479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e014f77f3c7647b188f015fffe90da8a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.239673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1006/1972] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:13.245298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:13.245298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:13.270082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:13.326312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093e5a0-6c35-4bfe-bce2-9bb275b1bd82 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:13.333546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e014f77f3c7647b188f015fffe90da8a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.334424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e014f77f3c7647b188f015fffe90da8a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.335271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e014f77f3c7647b188f015fffe90da8a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.336204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:13.337097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:13.337454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:13.337765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] PATH_INFO: `/v3/projects/2752f65b5af5411cb7978f470d14ac42` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:13.339211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2752f65b5af5411cb7978f470d14ac42)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:13.344333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:13.359647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 967/1973] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:12 2022] DELETE /identity/v3/users/e014f77f3c7647b188f015fffe90da8a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:13.446510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:13.448017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:13.485647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2752f65b5af5411cb7978f470d14ac42', 'request_id': 'req-8466b752-328b-488d-b736-7c092e0cbde6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.511148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2752f65b5af5411cb7978f470d14ac42', 'request_id': 'req-8466b752-328b-488d-b736-7c092e0cbde6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.549194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eb316-213a-4dba-ba72-0751eec7b352 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:13.600598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2752f65b5af5411cb7978f470d14ac42', 'request_id': 'req-8466b752-328b-488d-b736-7c092e0cbde6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.614267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:13.615080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:13.615370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:13.615656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] PATH_INFO: `/v3/users/ac601c4fb5784ff4ace1685ba7283f8f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:13.617120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac601c4fb5784ff4ace1685ba7283f8f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:13.617217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] The token cache is being invalidate because project 2752f65b5af5411cb7978f470d14ac42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:13.617525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.619001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.619150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:13.620788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1007/1974] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:13 2022] DELETE /identity/v3/projects/2752f65b5af5411cb7978f470d14ac42 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:13.626634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:13.630391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:13.647082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b752-328b-488d-b736-7c092e0cbde6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:13.653590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:13.654303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:13.654729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:13.655015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] PATH_INFO: `/v3/users/54cfca53919a42b2a420512b60d43b36` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:13.656976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cfca53919a42b2a420512b60d43b36)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:13.659131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:13.671832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:13.965333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac601c4fb5784ff4ace1685ba7283f8f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.966489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cfca53919a42b2a420512b60d43b36'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.967223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cfca53919a42b2a420512b60d43b36'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:13.978207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac601c4fb5784ff4ace1685ba7283f8f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.087096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cfca53919a42b2a420512b60d43b36'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.087762 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cfca53919a42b2a420512b60d43b36'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.100829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cfca53919a42b2a420512b60d43b36'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.115435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1008/1975] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:13 2022] DELETE /identity/v3/users/54cfca53919a42b2a420512b60d43b36 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:14.116084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.117670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.195369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb13a6-f9c0-4af4-9a06-70b7b80c13dd admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.200750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.201673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.201977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.202241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] PATH_INFO: `/v3/users/4c56d2fb57a04e69a5e5a672e55f47cb` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.203848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c56d2fb57a04e69a5e5a672e55f47cb)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.205988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:14.206566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac601c4fb5784ff4ace1685ba7283f8f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.207320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac601c4fb5784ff4ace1685ba7283f8f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.208005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac601c4fb5784ff4ace1685ba7283f8f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.210626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:14.223440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 968/1976] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:13 2022] DELETE /identity/v3/users/ac601c4fb5784ff4ace1685ba7283f8f => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:14.224447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.225859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.295038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908db6c-d080-4d3e-b2aa-d4dac61e8700 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.305050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.306022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.306311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.306592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] PATH_INFO: `/v3/projects/45e26d60656942d78e9d6a6eb932f190` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.307787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e26d60656942d78e9d6a6eb932f190)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.309821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:14.385739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c56d2fb57a04e69a5e5a672e55f47cb'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.386413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c56d2fb57a04e69a5e5a672e55f47cb'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.418643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e26d60656942d78e9d6a6eb932f190', 'request_id': 'req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.463778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c56d2fb57a04e69a5e5a672e55f47cb'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.464379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c56d2fb57a04e69a5e5a672e55f47cb'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.479405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e871c87a-83bb-4c89-b8a2-dce095f76e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c56d2fb57a04e69a5e5a672e55f47cb'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.493557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1009/1977] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:14 2022] DELETE /identity/v3/users/4c56d2fb57a04e69a5e5a672e55f47cb => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:14.495000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bad4-9721-47c7-801a-324432bafadb None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.495292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bad4-9721-47c7-801a-324432bafadb None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.495570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bad4-9721-47c7-801a-324432bafadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.510940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:14.517865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-02b0bad4-9721-47c7-801a-324432bafadb None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:14.519175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-02b0bad4-9721-47c7-801a-324432bafadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.521109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e26d60656942d78e9d6a6eb932f190', 'request_id': 'req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.528999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e26d60656942d78e9d6a6eb932f190', 'request_id': 'req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.544226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] The token cache is being invalidate because project 45e26d60656942d78e9d6a6eb932f190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:14.544543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.545249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6fb9b48d-58ea-49f1-92db-a4e2ed01475b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:14.546905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 969/1978] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:14 2022] DELETE /identity/v3/projects/45e26d60656942d78e9d6a6eb932f190 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:14.551974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe97d-5e72-4003-9d1d-d24309e706e2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.552252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe97d-5e72-4003-9d1d-d24309e706e2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.552840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbe97d-5e72-4003-9d1d-d24309e706e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.573229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-73cbe97d-5e72-4003-9d1d-d24309e706e2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:14.574734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-73cbe97d-5e72-4003-9d1d-d24309e706e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.592194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1010/1979] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:14.594745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675700ae-6992-443d-b265-364cd48ee359 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.595426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675700ae-6992-443d-b265-364cd48ee359 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.596232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675700ae-6992-443d-b265-364cd48ee359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.620448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-675700ae-6992-443d-b265-364cd48ee359 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:14.621733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-675700ae-6992-443d-b265-364cd48ee359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.631915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1011/1980] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:14.633088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675700ae-6992-443d-b265-364cd48ee359 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.647948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-675700ae-6992-443d-b265-364cd48ee359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.648156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 970/1981] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:14.649591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.649886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.650170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.673278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:14.674682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.683086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 971/1982] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:14.689294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.691112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:14.735055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675700ae-6992-443d-b265-364cd48ee359 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.740856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.741817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.742303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.742864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.770305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.772256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf0e01-c366-44b6-90ad-59078f4dd025 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.772787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:14.777357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.778724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.779197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.779746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.787508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1012/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:14.788546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.798023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.800187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:14.800880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1184-8eaf-4832-8848-189fcbbdea11 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.814903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 972/1984] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:14.819955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.820700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.820925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.821207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.821988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:14.826033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.826033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:14.838690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1013/1985] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:14.843893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:14.856038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ace43-2b3a-440e-89d3-bc12c2c5b199 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:14.863449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:14.864110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:14.864399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:14.864753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:14.865512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:14.867654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.100100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.101868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:15.105733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 973/1986] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:14 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:15.111349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.127471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f8b-05ad-4195-bb5c-03e75315a7a8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.179047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:15.241615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5a641-f22b-4f39-813c-b95bfd566c75 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.242859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.243770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.244063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.244353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] PATH_INFO: `/v3/projects/0a098ad584924616819dfe1cc43af023` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.245676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a098ad584924616819dfe1cc43af023)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.249660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.249778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.249778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.249778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.249778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.250774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.251878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.362214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 974/1987] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:15 2022] POST /identity/v3/users => generated 291 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:15.367087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.386381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008d70-d8b4-481e-a445-ffaa3e7eb7f3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.400572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.401288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.401631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.401959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.402750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.404448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.416311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a098ad584924616819dfe1cc43af023', 'request_id': 'req-47d79dc3-2d4e-431a-93e7-54c53d1c0665'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.417271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 975/1988] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:15.421024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.424071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a098ad584924616819dfe1cc43af023', 'request_id': 'req-47d79dc3-2d4e-431a-93e7-54c53d1c0665'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.433736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5786a43-4785-49ed-8067-412e3905a5bc admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.510982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.511680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.511966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.512244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.512974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.514714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.525846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 976/1989] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:15.526648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.528328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:15.541068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1736-63db-49f4-be2d-99e7c343b8fd admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.546082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.547281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.547610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.547924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] PATH_INFO: `/v3/projects/cd7f849538d44293999c28b49a00bed8` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.549157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7f849538d44293999c28b49a00bed8)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.551124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.555170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a098ad584924616819dfe1cc43af023', 'request_id': 'req-47d79dc3-2d4e-431a-93e7-54c53d1c0665'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.567677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] The token cache is being invalidate because project 0a098ad584924616819dfe1cc43af023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:15.567969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.568624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.570252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1014/1990] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:15 2022] DELETE /identity/v3/projects/0a098ad584924616819dfe1cc43af023 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:15.571014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.572326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:15.585878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d79dc3-2d4e-431a-93e7-54c53d1c0665 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.591828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.592764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.593044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.593415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] PATH_INFO: `/v3/projects/aeb387454a344814b3f187efe3cc95d7/users/a5ed51da9d4e4be8ba487a93c7ca4ab3/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.613017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb387454a344814b3f187efe3cc95d7, user_id=a5ed51da9d4e4be8ba487a93c7ca4ab3, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.615822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476ab01-b14e-4f77-8865-b3849b5366ba admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.761948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1015/1991] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:15 2022] PUT /identity/v3/projects/aeb387454a344814b3f187efe3cc95d7/users/a5ed51da9d4e4be8ba487a93c7ca4ab3/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:15.763255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.763534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.764160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.772237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7f849538d44293999c28b49a00bed8', 'request_id': 'req-6dba4e33-a475-47f8-a9bc-1c18527f83b0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.783340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:15.784593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:15.853492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1016/1992] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:15.853492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:15.853492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:15.861106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfe445-1ac0-4e07-87d1-7755e1dc4619 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:15.866161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:15.867029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.867320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.867603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] PATH_INFO: `/v3/projects/3bef23c708304098b2e6912aae0ef364` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.868830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bef23c708304098b2e6912aae0ef364)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:15.870876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:15.881935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:15.938723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7f849538d44293999c28b49a00bed8', 'request_id': 'req-6dba4e33-a475-47f8-a9bc-1c18527f83b0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.945525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7f849538d44293999c28b49a00bed8', 'request_id': 'req-6dba4e33-a475-47f8-a9bc-1c18527f83b0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.958643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] The token cache is being invalidate because project cd7f849538d44293999c28b49a00bed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:15.958943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.959603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6dba4e33-a475-47f8-a9bc-1c18527f83b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:15.961203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 977/1993] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:15 2022] DELETE /identity/v3/projects/cd7f849538d44293999c28b49a00bed8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:15.962521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb623e-5a72-4ede-983a-16a3e4773168 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:15.962759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb623e-5a72-4ede-983a-16a3e4773168 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:15.963259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb623e-5a72-4ede-983a-16a3e4773168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:15.995036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4beb623e-5a72-4ede-983a-16a3e4773168 None None] MFA Rules not processed for user `a5ed51da9d4e4be8ba487a93c7ca4ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:15.996441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4beb623e-5a72-4ede-983a-16a3e4773168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 978/1994] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3adb7-afec-456e-a42d-542fd0afc064 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3adb7-afec-456e-a42d-542fd0afc064 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3adb7-afec-456e-a42d-542fd0afc064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7de3adb7-afec-456e-a42d-542fd0afc064 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:16.103889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7de3adb7-afec-456e-a42d-542fd0afc064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.104608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 979/1995] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.104608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.104608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.104608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.118345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:16.119840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.143872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bef23c708304098b2e6912aae0ef364', 'request_id': 'req-b125535f-dd99-4508-ad32-4a42c99da149'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:16.155486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bef23c708304098b2e6912aae0ef364', 'request_id': 'req-b125535f-dd99-4508-ad32-4a42c99da149'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:16.194830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 980/1996] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.195659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.196910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.209140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5ef9-42a8-4dad-a6b2-b86bd3a63c9f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.213878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.214532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.214817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.215090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.215781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.217818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.470028 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bef23c708304098b2e6912aae0ef364', 'request_id': 'req-b125535f-dd99-4508-ad32-4a42c99da149'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:16.480925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 981/1997] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:16.480925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.482201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] The token cache is being invalidate because project 3bef23c708304098b2e6912aae0ef364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:16.482504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:16.483147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b125535f-dd99-4508-ad32-4a42c99da149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:16.485506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1017/1998] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:15 2022] DELETE /identity/v3/projects/3bef23c708304098b2e6912aae0ef364 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:16.487074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.487379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.487708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.507527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:16.513960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.552879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eee790-d27c-4c2d-90d8-987cb601a890 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.560339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.561460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.561933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.562644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.563915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:16.564446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.567381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6644ccf-1567-431e-a410-e0d6614b625e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.581556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1018/1999] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.584226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 982/2000] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:16.585771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.587461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.589036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.589547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.589974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.601366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7636d9f-9411-4a19-9e0c-848a97ec8be9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.606289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.607120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.607546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.607892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.608593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.611062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:16.611205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4969a7c-d7bc-4f34-9ec3-5a237d47b3d9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.612599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.620798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 983/2001] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.622433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.633018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebcf13-dfbb-468f-a5a1-70fecd4a6b5b None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.637829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.638478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.638765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.639044 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.639814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:16.640124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.643015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.655143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 984/2002] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:16.657060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.659106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.672544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733de150-4c4c-4dc4-a18b-70b53b920391 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.677388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.683803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.683803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.683803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.683803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.683803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.710586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1019/2003] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:16.712460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.712460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.712654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.746848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:16.746848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:16.748800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1020/2004] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:16.749597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.759126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808c04a-61f2-4584-8765-dd5fb434c0bf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.763936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.764655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.764943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.765219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.766602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.768519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.786957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 985/2005] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:16.787880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.853470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db360b-cdff-40ff-bd2e-dd3d7ddd57f0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.858427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.860461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.860461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.860461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.860461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.862486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.872792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 986/2006] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:16.873805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.883776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90586-e4d0-4429-ae7e-0a5e15e6f2af admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.889463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1021/2007] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:16.889631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.889631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.889828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.889908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.889908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.891372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:16.891549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.893262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.913725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 987/2008] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:16.914629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:16.924873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b6489-d746-4194-92fa-5e31cbdf202f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.959131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bafb64-ebb3-4102-a606-48049058628c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:16.964019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.964690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.964975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.965254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:16.965976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:16.967912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:16.988003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:16.992183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:16.992183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:16.992183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] PATH_INFO: `/v3/projects/504bb03d9d5641a7aa374733e2caafc3/users/4ae39aa81cfb4b609a0d4f9393d988ba/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.008199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504bb03d9d5641a7aa374733e2caafc3, user_id=4ae39aa81cfb4b609a0d4f9393d988ba, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.011097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.129457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 988/2009] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:16 2022] PUT /identity/v3/projects/504bb03d9d5641a7aa374733e2caafc3/users/4ae39aa81cfb4b609a0d4f9393d988ba/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:17.133456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.133909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1022/2010] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:16 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:17.135107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.146869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b046b-da1e-4837-8c18-61d417040281 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.209275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af86222-e8f0-49d1-a9bd-01dae132d154 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.214489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.215403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.215693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.215972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.216670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.218792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.222742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.223432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.223722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.224005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.224739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.226987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.259187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:17.367753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 989/2011] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:17 2022] POST /identity/v3/users => generated 294 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:17.376340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.381100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde974f6-35ec-41c2-b00a-d5b255da7ffe admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.386201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.386911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.387206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.387493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.388257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.390085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.401623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 990/2012] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:17.402778 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.413367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e3fdf-8c52-4be3-8c6a-3df60050b4a7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.418992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.419656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.419949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.420226 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.420944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.423124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.434901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 991/2013] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:17.435859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.443380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:17.447281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890025a5-6fc9-4378-a5fd-c31d9781f5ed admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.453089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1023/2014] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:17 2022] POST /identity/v3/projects => generated 363 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:17.453089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.470440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.470440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.470440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.470440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.470829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.470829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.478219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 992/2015] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:17.478219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.490771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae39d34-a7ef-46dc-8d88-7bcfaf3786a9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.527811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ba429-47a9-4430-88e6-78b60d308c66 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.533171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.534373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.534749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.535046 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] PATH_INFO: `/v3/projects/504bb03d9d5641a7aa374733e2caafc3/users/4ae39aa81cfb4b609a0d4f9393d988ba/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.537044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=504bb03d9d5641a7aa374733e2caafc3, user_id=4ae39aa81cfb4b609a0d4f9393d988ba, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.540020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.556744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1024/2016] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:17 2022] PUT /identity/v3/projects/504bb03d9d5641a7aa374733e2caafc3/users/4ae39aa81cfb4b609a0d4f9393d988ba/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:17.558104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.563246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.563997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.564250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.564537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.565327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.567150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.579369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 993/2017] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:17.580284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.590739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9f103-1a0a-462d-b7f6-b06c131b94cf admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.627593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff4bed4-159d-4a9c-b0af-30f232806644 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.632354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.633017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.633300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.633605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.634284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.636342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.653248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.654002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.654323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.654604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.655323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.657484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb01b1-99fa-4632-aacd-44f926a94c9a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.667292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 994/2018] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:17.668834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.669151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.669474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.698233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] MFA Rules not processed for user `4ae39aa81cfb4b609a0d4f9393d988ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:17.699520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:17.784304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 995/2019] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:17.785105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.796420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f44eb-62e2-448e-a63a-7f8a087a0200 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.809403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.810418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.810750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.811119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] PATH_INFO: `/v3/projects/aac437473fae4632840f5089550fb84f/users/d34c1c54534d4eaa8cc50425066b7618/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.829164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac437473fae4632840f5089550fb84f, user_id=d34c1c54534d4eaa8cc50425066b7618, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.834157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:17.928826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1025/2020] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:17 2022] POST /identity/v3/users => generated 282 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:17.933299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:17.940934 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593be7ba-a830-46c6-bbeb-171cdb15aeef admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:17.946087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:17.947159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:17.947451 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:17.947783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] PATH_INFO: `/v3/projects/7248207892454e728f302d95f7b7974a/users/0aa278e9651b4c078690061fcc899988/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:17.964053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7248207892454e728f302d95f7b7974a, user_id=0aa278e9651b4c078690061fcc899988, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:17.967005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.060621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 996/2021] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:17 2022] PUT /identity/v3/projects/aac437473fae4632840f5089550fb84f/users/d34c1c54534d4eaa8cc50425066b7618/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:18.061477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.065267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:18.121395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1026/2022] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:17 2022] PUT /identity/v3/projects/7248207892454e728f302d95f7b7974a/users/0aa278e9651b4c078690061fcc899988/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:18.122532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.135412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384dd34c-a0bb-452a-a0b4-853f55f08d4e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.187569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.188296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.188579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.188857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.198417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b0179-a26b-4bdc-a347-253908f5a8b3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.203957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.204887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.205171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.205481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.206220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.207846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084309a1-fdb6-47c5-b805-ca58e7fefc77 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.222044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1027/2023] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:18.222044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23477ab-9aca-4029-a984-05234d815301 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.222044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23477ab-9aca-4029-a984-05234d815301 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.222044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23477ab-9aca-4029-a984-05234d815301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.242866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a23477ab-9aca-4029-a984-05234d815301 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:18.243719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a23477ab-9aca-4029-a984-05234d815301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:18.252829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1028/2024] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:18.255154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.255154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.255154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.258119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.259238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.272340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 997/2025] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:18.273183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.286018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a446d-073f-4f58-b057-aad01a53be2b service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.289880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] MFA Rules not processed for user `0aa278e9651b4c078690061fcc899988`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:18.294266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.294422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:18.294520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.294520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.294520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.294520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.294942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.306051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 998/2026] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:18.306709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.308362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:18.362904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1029/2027] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:18.364121 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.371188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2502c9-16f3-42f9-97ad-76d4bdbc485e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.375745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.385061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.385061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.385276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a2777-d0d9-466f-b3db-af000bae6502 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.385395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.385474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.385474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.385474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.385474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] PATH_INFO: `/v3/projects/7bbf24de48a64da48640a0d42ff0f2ca/users/89ba90ccf07f492fa121f0bb916c2870/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.387413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.388925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.398796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bbf24de48a64da48640a0d42ff0f2ca, user_id=89ba90ccf07f492fa121f0bb916c2870, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.399728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 999/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:18.400671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.401721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a78df1-4719-44ff-b610-05f00d1ae96c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.411002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a18f1d-e33a-468f-a832-06281c1cae50 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.422910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.422910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.422910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.422910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.423402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.425417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.525574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1030/2029] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:18 2022] PUT /identity/v3/projects/7bbf24de48a64da48640a0d42ff0f2ca/users/89ba90ccf07f492fa121f0bb916c2870/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:18.541800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.541800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.541800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.543377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1000/2030] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:18 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:18.545690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.597959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] MFA Rules not processed for user `89ba90ccf07f492fa121f0bb916c2870`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:18.599532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:18.691459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e04a0-9d4a-47de-a1ae-280d14cb8561 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.705213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1031/2031] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:18.715816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.715816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.715816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.715816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.716836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.716836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.727561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.801651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b54a8-b775-4d3d-bb70-7e083fb480ca None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.808353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.808353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.808353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.808353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.816792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.818392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.824142 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:18.834190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1032/2032] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:18.845587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1001/2033] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:18 2022] POST /identity/v3/users => generated 299 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:18.851540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.907126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac6d9f-80e8-42ed-96dd-70ad4181bdae service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.929717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.930552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.930983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.931294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:18.932320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:18.934163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:18.959010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1033/2034] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:18.959791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:18.981956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca69dcb-bb47-4798-b62e-8fc423a605ba admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:18.992393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:18.993414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:18.993690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:18.994023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] PATH_INFO: `/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d/users/b9456a84dfc04481975d59723d914320/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.010942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b92499cf9bd4c5eaed7ea4b78a3740d, user_id=b9456a84dfc04481975d59723d914320, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.013681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.134442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1002/2035] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:18 2022] PUT /identity/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d/users/b9456a84dfc04481975d59723d914320/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:19.147822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.148667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.159237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627673a0-6932-4c96-a76e-6936b150663b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.246524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.246839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.247210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.247557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.248301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.250927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.262339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a806291-b6e2-4457-bcb5-232c210b6b70 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.264100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1034/2036] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:19.269117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.279286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.279286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.279286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.279286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] PATH_INFO: `/v3/users/a5ed51da9d4e4be8ba487a93c7ca4ab3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.279783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ed51da9d4e4be8ba487a93c7ca4ab3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.279783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.280209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f1cea-ec50-474f-9ce5-20f6e60d24b9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.288203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.288666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.289399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.290097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] PATH_INFO: `/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d/users/b9456a84dfc04481975d59723d914320/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.292420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b92499cf9bd4c5eaed7ea4b78a3740d, user_id=b9456a84dfc04481975d59723d914320, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.296147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db59d-dcbe-4084-a865-8263aee6dd96 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.310951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1035/2037] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:19 2022] PUT /identity/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d/users/b9456a84dfc04481975d59723d914320/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:19.324956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.325286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.325639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.359082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] MFA Rules not processed for user `b9456a84dfc04481975d59723d914320`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:19.361326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:19.426705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:19.446238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1036/2038] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:19.461120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.463416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:19.537501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2778ac0-09d7-4ebb-ad7e-415705604355 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.541444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.542117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.542402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.542802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.546992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ed51da9d4e4be8ba487a93c7ca4ab3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:19.553712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.555271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.562866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ed51da9d4e4be8ba487a93c7ca4ab3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:19.569111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1037/2039] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:19.643602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.693185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ed51da9d4e4be8ba487a93c7ca4ab3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:19.697198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ed51da9d4e4be8ba487a93c7ca4ab3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:19.706233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2eda98d6-7543-4953-b6f1-7b7de8baa140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ed51da9d4e4be8ba487a93c7ca4ab3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:19.725281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1003/2040] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:19 2022] DELETE /identity/v3/users/a5ed51da9d4e4be8ba487a93c7ca4ab3 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:19.727518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.727904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.728276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.728983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1a23-714a-4d51-bfe3-3beb40bf3937 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.749193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:19.751851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:19.857538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.862106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.862106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.862106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] PATH_INFO: `/v3/users/0aa278e9651b4c078690061fcc899988` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.862106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa278e9651b4c078690061fcc899988)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.863971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.869489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1004/2041] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:19.879743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:19.881423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:19.949300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bea8f-6bd6-4f7a-8810-c2fa41b03b3d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:19.953912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:19.954780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:19.955070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:19.955341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:19.965372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:19.967085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115d8c3-56b9-4fbf-959d-13f3af90358b service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:19.977802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1005/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:20.221278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa278e9651b4c078690061fcc899988'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.222110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa278e9651b4c078690061fcc899988'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.237076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:20.305190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa278e9651b4c078690061fcc899988'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.309099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa278e9651b4c078690061fcc899988'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.322958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-3e815e40-b0c0-41d1-b3f0-a80deb8151d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa278e9651b4c078690061fcc899988'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.349770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1038/2043] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:19 2022] DELETE /identity/v3/users/0aa278e9651b4c078690061fcc899988 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:20.354949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.355249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.355808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.378791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:20.380278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:20.412124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.412915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.414035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.420225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:20.479033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] MFA Rules not processed for user `d34c1c54534d4eaa8cc50425066b7618`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:20.480188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:20.496901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1006/2044] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:20.506535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:20.508861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:20.575862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1039/2045] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:20.577465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:20.582974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:20.597829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590e636-62b6-4cc1-8b13-6322e3ca53e5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:20.602861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:20.603721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.604012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.604299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] PATH_INFO: `/v3/projects/aeb387454a344814b3f187efe3cc95d7` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.605632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8049e84-114f-4fb1-b8ee-5bd7264e51a6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:20.605756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb387454a344814b3f187efe3cc95d7)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:20.607781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:20.612020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:20.612838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.613195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.613625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.624624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:20.626349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:20.639490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1007/2046] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:20.645946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:20.645946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:20.658847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057f7a-aeb1-48c4-bc09-88971b1bd172 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:20.685052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:20.686335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.697721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.697721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.697721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:20.697721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:20.799917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb387454a344814b3f187efe3cc95d7', 'request_id': 'req-e306fac3-9144-4187-b185-bad6eed7e4f7'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.805808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb387454a344814b3f187efe3cc95d7', 'request_id': 'req-e306fac3-9144-4187-b185-bad6eed7e4f7'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:20.845250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1008/2047] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:20 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:20.848729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:20.913777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675fd21-429e-4a1d-adb9-c3933192d037 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:20.918818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:20.919506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:20.919833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:20.920408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:20.921168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:20.923803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.128626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb387454a344814b3f187efe3cc95d7', 'request_id': 'req-e306fac3-9144-4187-b185-bad6eed7e4f7'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.143023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1009/2048] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:20 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:21.148311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] The token cache is being invalidate because project aeb387454a344814b3f187efe3cc95d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:21.148485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.148569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.150332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.152065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1040/2049] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:20 2022] DELETE /identity/v3/projects/aeb387454a344814b3f187efe3cc95d7 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:21.158940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.160226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:21.165534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e2ed2-a2e3-438b-8cab-d11c0580e41c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.171117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.173006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.173800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.174329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.175273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.177313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.205138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1010/2050] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:21.205138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.206341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:21.226000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd67d79-7fc5-4253-aa04-7b5d5d46fad1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.231939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.232849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.233139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.233469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] PATH_INFO: `/v3/projects/b27dc5abdef34480942ca19891961563/users/ad66fd3c657c40fd8c45d630b42b797e/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.253169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27dc5abdef34480942ca19891961563, user_id=ad66fd3c657c40fd8c45d630b42b797e, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.256386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.257583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306fac3-9144-4187-b185-bad6eed7e4f7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.264725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.266450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.266823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.267255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] PATH_INFO: `/v3/users/4ae39aa81cfb4b609a0d4f9393d988ba` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.268642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae39aa81cfb4b609a0d4f9393d988ba)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.270707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.397654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1011/2051] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:21 2022] PUT /identity/v3/projects/b27dc5abdef34480942ca19891961563/users/ad66fd3c657c40fd8c45d630b42b797e/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:21.402880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.493687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b421a-f48b-4971-9626-2f101222c69e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.499110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.501641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.501641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.501641 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.509267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.509267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.515631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1012/2052] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:21.520160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.522854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:21.536712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5603f-6405-4462-8280-36f05ad2aac0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.543446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.544324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.544661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.552613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] PATH_INFO: `/v3/projects/7248207892454e728f302d95f7b7974a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.552613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7248207892454e728f302d95f7b7974a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.552613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.554885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae39aa81cfb4b609a0d4f9393d988ba'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.555590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae39aa81cfb4b609a0d4f9393d988ba'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.676222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae39aa81cfb4b609a0d4f9393d988ba'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.676843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae39aa81cfb4b609a0d4f9393d988ba'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.677413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:21.691110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae39aa81cfb4b609a0d4f9393d988ba'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.697868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7248207892454e728f302d95f7b7974a', 'request_id': 'req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.707131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1041/2053] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:21 2022] DELETE /identity/v3/users/4ae39aa81cfb4b609a0d4f9393d988ba => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:21.708035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.709497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:21.792448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f570b0-6578-4d74-bb6f-4b68239f13ec admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.806138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.807227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.807632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.808065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] PATH_INFO: `/v3/projects/b27dc5abdef34480942ca19891961563/users/ad66fd3c657c40fd8c45d630b42b797e/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.818198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27dc5abdef34480942ca19891961563, user_id=ad66fd3c657c40fd8c45d630b42b797e, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.818462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7248207892454e728f302d95f7b7974a', 'request_id': 'req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.818548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.821204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7248207892454e728f302d95f7b7974a', 'request_id': 'req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.829146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1042/2054] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:21 2022] PUT /identity/v3/projects/b27dc5abdef34480942ca19891961563/users/ad66fd3c657c40fd8c45d630b42b797e/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:21.840287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] The token cache is being invalidate because project 7248207892454e728f302d95f7b7974a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:21.840627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.841598 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:21.846872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1013/2055] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:21 2022] DELETE /identity/v3/projects/7248207892454e728f302d95f7b7974a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:21.848966 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:21.852450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:21.929597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ebb5a-4253-40ae-bade-428967679eb2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:21.935430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:21.936362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:21.936611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:21.936892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] PATH_INFO: `/v3/users/b9456a84dfc04481975d59723d914320` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:21.938167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9456a84dfc04481975d59723d914320)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:21.940128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:21.977938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:22.251857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9456a84dfc04481975d59723d914320'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:22.252574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9456a84dfc04481975d59723d914320'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:22.345230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9456a84dfc04481975d59723d914320'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:22.346412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9456a84dfc04481975d59723d914320'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:22.360813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9456a84dfc04481975d59723d914320'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:22.379704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1043/2056] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:21 2022] DELETE /identity/v3/users/b9456a84dfc04481975d59723d914320 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:22.659223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:22.661311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:22.757653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5170-5cca-42f6-8e9b-dc3b386888fd admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:22.819664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:22.820676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:22.820980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:22.821275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] PATH_INFO: `/v3/projects/504bb03d9d5641a7aa374733e2caafc3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:22.822712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504bb03d9d5641a7aa374733e2caafc3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:22.824869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:22.941755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:22.944447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:22.962784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634370-88ca-4548-aad5-57463c743345 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:22.972241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:22.973117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:22.973442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:22.973728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] PATH_INFO: `/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:22.975030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b92499cf9bd4c5eaed7ea4b78a3740d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:22.976870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:22.996075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:23.018028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504bb03d9d5641a7aa374733e2caafc3', 'request_id': 'req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.174673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504bb03d9d5641a7aa374733e2caafc3', 'request_id': 'req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.181794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504bb03d9d5641a7aa374733e2caafc3', 'request_id': 'req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.194663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] The token cache is being invalidate because project 504bb03d9d5641a7aa374733e2caafc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:23.194843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.195480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a7823ea8-d8db-4eb8-b9e5-eeb66404bc19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.197319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1014/2057] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:22 2022] DELETE /identity/v3/projects/504bb03d9d5641a7aa374733e2caafc3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:23.200813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b92499cf9bd4c5eaed7ea4b78a3740d', 'request_id': 'req-7918b600-ef21-44d3-978e-d434b03690aa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.202823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf591d-828a-4492-87f2-5f6431dcef2a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.203176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf591d-828a-4492-87f2-5f6431dcef2a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.203574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf591d-828a-4492-87f2-5f6431dcef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.206987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b92499cf9bd4c5eaed7ea4b78a3740d', 'request_id': 'req-7918b600-ef21-44d3-978e-d434b03690aa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.225571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4baf591d-828a-4492-87f2-5f6431dcef2a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:23.226823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4baf591d-828a-4492-87f2-5f6431dcef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:23.293568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b92499cf9bd4c5eaed7ea4b78a3740d', 'request_id': 'req-7918b600-ef21-44d3-978e-d434b03690aa'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.307615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] The token cache is being invalidate because project 7b92499cf9bd4c5eaed7ea4b78a3740d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:23.308123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.309292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7918b600-ef21-44d3-978e-d434b03690aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:23.311312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1044/2058] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:22 2022] DELETE /identity/v3/projects/7b92499cf9bd4c5eaed7ea4b78a3740d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:23.316189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f2b3-03fc-4a76-8990-9636e5c6091b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.316377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1015/2059] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:23.317127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f2b3-03fc-4a76-8990-9636e5c6091b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.317440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f2b3-03fc-4a76-8990-9636e5c6091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.323862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.324157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.324438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.338010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1ac4f2b3-03fc-4a76-8990-9636e5c6091b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:23.339247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1ac4f2b3-03fc-4a76-8990-9636e5c6091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:23.342883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:23.344655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:23.348912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1045/2060] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:23.352711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1016/2061] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:23.357124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.357270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.357458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.357748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.366857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26db9c-b7f9-408d-a5a5-4d1da8b0eece None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.371435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.372085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.372366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.372633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.373366 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:23.373694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.375354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.377546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:23.378818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:23.386978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1046/2062] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:23.387454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1017/2063] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:30:23.392245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.392389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.401541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554b0b3-2017-4cb8-b29d-caf67943c6f6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.404132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b90db-0f37-40b6-b243-9f4abb26034a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.408102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.409089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.409619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.410081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.411116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:23.411536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.412573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.413251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.413503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.413683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.414097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.414845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.416932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.434449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1047/2064] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:23.440593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.450987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50598e4-41db-4760-8c31-7d370823343b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.456780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.469678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.469678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.469678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.469678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.469678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.499551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:23.552227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1018/2065] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:23.556164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.619205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06347360-8cf4-4c89-9338-a31179bd6e16 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.630096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.630865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.631157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.631441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.632155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.634196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.660692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1048/2066] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:23.664765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.750794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb548-b503-42c3-97fc-828a35d3d421 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.755961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.756642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.756931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.757211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.757994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.760180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.786111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1019/2067] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:23.790269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.802259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74969f82-3e44-46f4-9ab8-2ce55cc723cb admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.810223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.813988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.813988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.813988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.814683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.816225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8db25-b726-4b2f-9727-9927bcbd264b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.842864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1020/2068] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:23.842864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.842864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.842864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.879284 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] MFA Rules not processed for user `ad66fd3c657c40fd8c45d630b42b797e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:23.881604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:23.909164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1049/2069] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:23.909164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.924154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0920cb-604c-48bc-a632-d50398c29afd admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.934544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.935361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.935871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.936385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.937571 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.940953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.960681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1050/2070] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:23.961657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:23.976301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83416f3f-520b-4198-ad60-f57797ee7adc admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:23.981249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:23.981956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:23.982265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:23.982551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:23.983289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:23.986277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:23.988932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1021/2071] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:23.989964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:24.003661 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ded43-4134-4d7d-b3f8-122b9619bb21 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:24.003891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1051/2072] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:24.004617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:24.006353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:24.013806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:24.015433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.015726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.016049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] PATH_INFO: `/v3/projects/4a2cb36edcb64a72800e4570a21eab58/users/d7cbe773ca624668a82add611a8ce46f/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.035022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2cb36edcb64a72800e4570a21eab58, user_id=d7cbe773ca624668a82add611a8ce46f, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:24.037615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:24.070621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fc74d-a8b5-49ea-a19a-bb68213588cb admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:24.074343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:24.074994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.075304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.075554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.077544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:24.134043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1022/2073] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:23 2022] PUT /identity/v3/projects/4a2cb36edcb64a72800e4570a21eab58/users/d7cbe773ca624668a82add611a8ce46f/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:24.134957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:24.152683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:24.154317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9818e-8bd0-409b-ac69-10d416f64df6 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:24.166024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1052/2074] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:24.174490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.174490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.174490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.208398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabecc9-6121-419b-8043-8bdc5db77e60 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:24.217819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:24.220306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.220945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.221242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.222148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] MFA Rules not processed for user `d7cbe773ca624668a82add611a8ce46f`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:24.222321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:24.223920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:24.224048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:24.240201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1023/2075] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:24.246765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:24.269042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a43d-3e8e-4fef-a226-2af07d5f1454 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:24.276660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:24.277721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.278010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.278291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] PATH_INFO: `/v3/projects/88f548281b9047168b4cb31f59293bdb/users/efe6fc664b26426381e3d7c49ea3c465/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.304989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f548281b9047168b4cb31f59293bdb, user_id=efe6fc664b26426381e3d7c49ea3c465, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:24.311581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b381d-ac28-4e1d-9eda-82a0187fb6a4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:24.312149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1053/2076] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:24.323796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:24.326185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:24.410509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57638e7-cbbf-4c71-bc39-d1dd99b1e2e7 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:24.414728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:24.415610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.415949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.416226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.420424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1024/2077] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:24 2022] PUT /identity/v3/projects/88f548281b9047168b4cb31f59293bdb/users/efe6fc664b26426381e3d7c49ea3c465/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:24.427452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:24.427630 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:24.432083 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:24.432384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:24.432782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:24.439238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1054/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:24.458546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:24.473853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] MFA Rules not processed for user `efe6fc664b26426381e3d7c49ea3c465`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:24.475580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:24.588553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1025/2079] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:24.962161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:25.249909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:25.260489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcf35-b16e-490a-b9d9-5e6b7911132e service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:25.318608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:25.320182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:25.320402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:25.320680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:25.336040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:25.337588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2a74b-402b-40fc-9231-6e40cfc1ef7e service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:25.349389 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1055/2080] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:25.371965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:25.479702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978b8c3-002a-4551-820d-360f8bfb86ec None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:25.485320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:25.486286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:25.486571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:25.486848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] PATH_INFO: `/v3/users/efe6fc664b26426381e3d7c49ea3c465` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:25.488403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe6fc664b26426381e3d7c49ea3c465)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:25.490437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:25.810125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe6fc664b26426381e3d7c49ea3c465'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:25.822619 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe6fc664b26426381e3d7c49ea3c465'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:25.950965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe6fc664b26426381e3d7c49ea3c465'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:25.951606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe6fc664b26426381e3d7c49ea3c465'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:25.952236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe6fc664b26426381e3d7c49ea3c465'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:25.965907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1026/2081] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:25 2022] DELETE /identity/v3/users/efe6fc664b26426381e3d7c49ea3c465 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:25.971325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:25.971843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:25.971939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:25.990703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:25.991936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:26.051885 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1056/2082] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:26.061836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:26.063179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:26.126851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57379-b0a1-4b68-b635-d21abe0e1504 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:26.130429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:26.131180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:26.131443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:26.131720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:26.140662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:26.142162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:26.151756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1027/2083] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:26.500942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:26.514225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:26.516882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:26.585039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b4f9-92b3-412a-bead-ddaa53d29f4d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:26.588851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:26.589606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:26.589895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:26.590707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:26.592127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:26.684949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:26.688863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1255404-46ac-41bf-8356-2aa981127366 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:26.704777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1057/2084] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:26.734255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:26.736069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:26.773931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4faee05-3469-4607-a3ff-4ae850dbca19 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:26.782919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:26.784123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:26.784486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:26.784809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] PATH_INFO: `/v3/projects/88f548281b9047168b4cb31f59293bdb` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:26.786670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f548281b9047168b4cb31f59293bdb)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:26.789057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:26.956759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f548281b9047168b4cb31f59293bdb', 'request_id': 'req-47573b5e-fbdd-4567-a9e9-a26b48dd6937'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:27.089064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f548281b9047168b4cb31f59293bdb', 'request_id': 'req-47573b5e-fbdd-4567-a9e9-a26b48dd6937'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:27.097744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f548281b9047168b4cb31f59293bdb', 'request_id': 'req-47573b5e-fbdd-4567-a9e9-a26b48dd6937'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:27.110903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] The token cache is being invalidate because project 88f548281b9047168b4cb31f59293bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:27.111218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:27.111999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-47573b5e-fbdd-4567-a9e9-a26b48dd6937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:27.117190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1028/2085] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:26 2022] DELETE /identity/v3/projects/88f548281b9047168b4cb31f59293bdb => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:27.118792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e498-c2f6-4296-a298-b624bba309be None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.119252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e498-c2f6-4296-a298-b624bba309be None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.119419 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e498-c2f6-4296-a298-b624bba309be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.149131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b325e498-c2f6-4296-a298-b624bba309be None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:27.150425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b325e498-c2f6-4296-a298-b624bba309be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:27.214012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1058/2086] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:27.225120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.225770 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.226153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.246319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:27.247557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:27.255912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1029/2087] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:27.260980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325e498-c2f6-4296-a298-b624bba309be None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.273417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325e498-c2f6-4296-a298-b624bba309be None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.281509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.282985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.283344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.283723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.284563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:27.285255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.286923 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.301271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1059/2088] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:27.306810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.329961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8ed4e-e0bd-4b30-96d2-ba882220c2a5 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.334909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.335648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.336000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.336384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.337159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.339295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.433256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1030/2089] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:27 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:27.437311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.447800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6bcb7-d038-4fc6-88b1-943cf2e6abf1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.511470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.512068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.512379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.512654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.513390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.515401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.631371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:27.671945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1060/2090] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:27 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:27.680272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.693061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44749-153f-45ac-a5c8-598624e1aab9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.698780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.699564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.700081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.700516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.701597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.703301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.721875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1031/2091] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:27.725517 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.740562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d761e24-beaa-4f4a-97d4-94a863c58cd0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.746265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.752659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.752659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.752659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.752659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.752659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e493e0a-b5f8-49fa-8e68-90d68f0d98d4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.765708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1061/2092] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:27.767656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:27.778638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b179-0167-42be-b82e-cc9bd8a14d7a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:27.793232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:27.794195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.794637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.795073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] PATH_INFO: `/v3/projects/4483daf6876c4f44a6cd6e281b35c16b/users/cf8009de53a2414b86eae128241a1be2/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:27.820792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4483daf6876c4f44a6cd6e281b35c16b, user_id=cf8009de53a2414b86eae128241a1be2, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:27.820792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:27.970033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1032/2093] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:27 2022] PUT /identity/v3/projects/4483daf6876c4f44a6cd6e281b35c16b/users/cf8009de53a2414b86eae128241a1be2/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:27.978867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:27.979171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:27.979462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.015066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] MFA Rules not processed for user `cf8009de53a2414b86eae128241a1be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:28.016293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:28.096403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1062/2094] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:28.100862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:28.171097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe99-c855-44d1-9d02-fa6c14fc05f7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:28.177438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:28.178189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.178479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.178751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.179437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:28.181357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:28.278690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1033/2095] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:28 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:28.283972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:28.295667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473967b-4808-4b76-9067-8b8d9a15c881 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:28.373078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:28.373849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.374156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.374438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.375189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:28.377878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:28.489859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1063/2096] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:28 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:28.494469 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:28.505585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22b1d8-452c-4fdf-ba24-8d6a38f2ac5b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:28.510821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:28.511490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.511779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.512054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.512832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:28.514476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:28.528662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1034/2097] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:28.533556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:28.544149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36685e57-29a9-47e2-afe5-bd3b0aedcb3c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:28.550404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:28.551084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.551497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.551786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.552687 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:28.554402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17859603-e2a3-4383-886a-b188dd8c1099 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:28.565906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1064/2098] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:28.570329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:28.582841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866f4e0-ec5f-4d6a-8204-608677359436 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:28.588481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:28.589579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.589726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.590010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] PATH_INFO: `/v3/projects/131464c100354ca7974dff994292c25c/users/306bda7b6ae44f15a4d120e3e0d87d3b/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.617164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] RBAC: Authorizing `identity:create_grant(project_id=131464c100354ca7974dff994292c25c, user_id=306bda7b6ae44f15a4d120e3e0d87d3b, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:28.617164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:28.823250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1035/2099] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:28 2022] PUT /identity/v3/projects/131464c100354ca7974dff994292c25c/users/306bda7b6ae44f15a4d120e3e0d87d3b/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:28.835388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:28.835681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:28.836018 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:28.866747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] MFA Rules not processed for user `306bda7b6ae44f15a4d120e3e0d87d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:28.868493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:28.943745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1065/2100] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:28.948909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.021398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926a497-3539-4513-9baa-275e842c458b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.027318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.028025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.028315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.028649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.031602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.032710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.191807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:29.204860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1036/2101] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:28 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:29.206724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.217649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cf57c-6d99-4905-b0cd-3000376c40f9 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.287083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.296385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.296385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.296385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.296385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.296385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.419307 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1066/2102] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:29 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:29.420562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.433287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62c42-7723-4ca2-8818-85d6d926f093 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.439800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.440507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.440954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.441150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.441973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.444032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.456875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1037/2103] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:29.461479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.473758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb2f8c-a5e5-4a2e-8238-56b2a4a86de5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.478548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.479436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.479768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.480155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] PATH_INFO: `/v3/projects/188df723b4464985a8046aec80f62775/users/53dcf6cdab86453a9d655261929e4767/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.505942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188df723b4464985a8046aec80f62775, user_id=53dcf6cdab86453a9d655261929e4767, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.509583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.675877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1067/2104] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:29 2022] PUT /identity/v3/projects/188df723b4464985a8046aec80f62775/users/53dcf6cdab86453a9d655261929e4767/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:29.676254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.687155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158958b2-f88a-4bd2-8bd3-394a5ce53399 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.754452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.755125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.755556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.755837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.756567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.758231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6647d-2c2b-4244-8b22-499a9d847210 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.768818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1038/2105] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:29.773126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.784872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fbfd1-a673-42de-ac8a-4aa5965be542 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:29.789808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:29.790688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.790970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.791242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] PATH_INFO: `/v3/projects/188df723b4464985a8046aec80f62775/users/53dcf6cdab86453a9d655261929e4767/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.793219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188df723b4464985a8046aec80f62775, user_id=53dcf6cdab86453a9d655261929e4767, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:29.795910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:29.809734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1068/2106] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:29 2022] PUT /identity/v3/projects/188df723b4464985a8046aec80f62775/users/53dcf6cdab86453a9d655261929e4767/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:29.822113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:29.822604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:29.823099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:29.858380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] MFA Rules not processed for user `53dcf6cdab86453a9d655261929e4767`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:29.861449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:29.946727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1039/2107] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:29.957763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:29.965226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:30.026358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485925bd-9bc5-4834-a1ed-8bbaee53eec9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:30.030396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:30.031104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:30.031391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:30.031732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:30.091216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:30.092685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74116def-fb6b-40ec-8bf9-17ef881d5ac9 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:30.103397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1069/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:30.293485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:30.671700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:31.831912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:31.935700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.294160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.363264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.568878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.654637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.906014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:32.951502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:33.043170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:33.151773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:33.180369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:33.379945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:34.170255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:34.171724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:34.241943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf4eb-77ab-40fa-af77-ca2d4f053373 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:34.247353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:34.248246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:34.248529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:34.248813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] PATH_INFO: `/v3/users/d7cbe773ca624668a82add611a8ce46f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:34.250074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7cbe773ca624668a82add611a8ce46f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:34.252324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:34.623249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7cbe773ca624668a82add611a8ce46f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:34.636794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7cbe773ca624668a82add611a8ce46f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:34.757315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7cbe773ca624668a82add611a8ce46f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:34.757968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7cbe773ca624668a82add611a8ce46f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:34.758611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7cbe773ca624668a82add611a8ce46f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:34.773473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1040/2109] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:34 2022] DELETE /identity/v3/users/d7cbe773ca624668a82add611a8ce46f => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:34.778072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:34.778530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:34.778620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:34.797174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:34.799261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:34.867919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1070/2110] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:34.878773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:34.880150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:34.965170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a3ba-6d2d-469b-a3db-586f76c63d27 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:34.969615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:34.970396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:34.970683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:34.970970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:34.983425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:34.985008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa33741-1e60-4052-a535-b6610db845da service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:35.000389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1041/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:35.733718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:35.733718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:35.760314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03692b1d-643e-4cae-b5b5-431f10d873d5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:35.769604 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:35.770750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:35.771041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:35.771667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] PATH_INFO: `/v3/projects/4a2cb36edcb64a72800e4570a21eab58` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:35.773189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2cb36edcb64a72800e4570a21eab58)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:35.776009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:35.918499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2cb36edcb64a72800e4570a21eab58', 'request_id': 'req-2395f454-6120-4474-89cc-f69f0ebd3004'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:35.924330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2cb36edcb64a72800e4570a21eab58', 'request_id': 'req-2395f454-6120-4474-89cc-f69f0ebd3004'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:36.054545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2cb36edcb64a72800e4570a21eab58', 'request_id': 'req-2395f454-6120-4474-89cc-f69f0ebd3004'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:36.067001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] The token cache is being invalidate because project 4a2cb36edcb64a72800e4570a21eab58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:36.067299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:36.070550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2395f454-6120-4474-89cc-f69f0ebd3004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:36.070550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1071/2112] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:35 2022] DELETE /identity/v3/projects/4a2cb36edcb64a72800e4570a21eab58 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:36.075407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.075947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.076071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.096227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:36.097484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:36.101947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:36.160461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1042/2113] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:36.169647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.169945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.170362 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.206603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:36.207837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:36.218229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1072/2114] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:36.228505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.239008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db475-b77d-4c75-8bbe-d582133f4b4f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.244037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.244513 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.244800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.245084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.246288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:36.246288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.247799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.271090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1043/2115] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:30:36.276020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.289617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba6a91-0bc8-4520-b630-ec9d7c0a75ab None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.306069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.306747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.307030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.307310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.308015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.310209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.362417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.364568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:36.427215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f40d8-2628-424b-958b-fbafe09701ab admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.431034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.431689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.431976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.432259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.434079 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:36.459346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1073/2116] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:36 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:36.463710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.498426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.499852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.509579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1044/2117] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:36.524532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:36.540351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d310e9-d385-4993-9053-a21309caa1a9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.546861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.547838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.548189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.549484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.550612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.552732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.670531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1074/2118] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:36 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:36.674879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.688668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab4173-b5c6-44d3-bd7b-575368efdad6 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.697819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.698553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.698973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.699135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.699834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.704052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.728329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1045/2119] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:36.732548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.746144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd09237-46ee-4936-adcc-ece70b9a96f5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.752459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.753587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.753948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.754237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.754983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.756809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935bdac-ccca-4f3b-8166-81afb9ffcccb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.770020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1075/2120] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:36.774245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:36.787967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fbc90-8423-43b4-904c-550b387790e9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:36.793383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:36.794416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.794552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.794828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] PATH_INFO: `/v3/projects/1e34322f6aa14465afc11a3842161882/users/c5c44bb198b24d73b15f3b6bf1027aec/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.815397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e34322f6aa14465afc11a3842161882, user_id=c5c44bb198b24d73b15f3b6bf1027aec, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:36.818830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:36.909186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1046/2121] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:36 2022] PUT /identity/v3/projects/1e34322f6aa14465afc11a3842161882/users/c5c44bb198b24d73b15f3b6bf1027aec/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:36.920171 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:36.920796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:36.920796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:36.951965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] MFA Rules not processed for user `c5c44bb198b24d73b15f3b6bf1027aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:36.953260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:37.022183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1076/2122] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:37.026756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:37.092875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73920d70-a944-4b59-bde1-7362316b6eb2 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:37.097860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:37.098520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.098807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.099093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.099801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:37.101811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:37.252868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1047/2123] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:37 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:37.253222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:37.265200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ba560-83fd-40cc-a931-6d74ac5f69ff None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:37.338223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:37.338974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.339259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.339545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.340243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:37.342281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:37.526608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1077/2124] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:37 2022] POST /identity/v3/users => generated 289 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:37.531370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:37.547665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cad48-d9fb-45b7-9595-ce15ec9d983e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:37.553398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:37.554120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.554411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.554692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.555420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:37.557040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:37.569107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1048/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:37.586275 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:37.591806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151178-8fe9-4da0-b1d2-46910bcb7b46 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:37.598761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:37.599864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.600328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.600791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.602257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:37.605026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531aebc-326e-49ef-b2a1-033208e10424 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:37.622302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1078/2126] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:37.623508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:37.635159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1aaaaa-3cf0-4ffc-9708-a692145fcc73 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:37.642715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:37.643713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.644560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.644560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] PATH_INFO: `/v3/projects/45c3b1f2cea34ce182bbfe61a7c450d7/users/8af62d95a1794c0bbf7855277236f3ea/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.672678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c3b1f2cea34ce182bbfe61a7c450d7, user_id=8af62d95a1794c0bbf7855277236f3ea, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:37.677533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:37.760540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:37.821678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1049/2127] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:37 2022] PUT /identity/v3/projects/45c3b1f2cea34ce182bbfe61a7c450d7/users/8af62d95a1794c0bbf7855277236f3ea/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:37.831980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:37.832272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:37.832555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:37.869718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] MFA Rules not processed for user `8af62d95a1794c0bbf7855277236f3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:37.872437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:38.023080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1079/2128] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:38.028009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.105257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55455a8-2628-45ba-a511-affd271be73a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.110117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.111269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.111657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.111657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.112669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.114872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.279905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1050/2129] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:38 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:30:38.284499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.295256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76cca-7caa-487a-b3a7-996e56e69be6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.306742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:38.364088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.364773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.365058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.365359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.366107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.368088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.474846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1080/2130] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:38 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:38.478102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.493746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b76adb-d71f-40f9-aaf9-d5ce3eae89b0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.501068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.504321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.504728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.505127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.506289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.510960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.522140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1051/2131] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:30:38.526830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.538123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520da6e-fbdd-4de6-8d96-39018478229b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.543095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.545527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.545527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.545527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] PATH_INFO: `/v3/projects/fe5c3c26811f4e87b873916872543522/users/c3f1719ccd134173adc3d41eafc7dd03/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.566493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c3c26811f4e87b873916872543522, user_id=c3f1719ccd134173adc3d41eafc7dd03, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.569436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.730791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1081/2132] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:38 2022] PUT /identity/v3/projects/fe5c3c26811f4e87b873916872543522/users/c3f1719ccd134173adc3d41eafc7dd03/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:38.735099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.750191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbededfb-531d-4017-9ccc-f45c103eaaf3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.795015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.855286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.856361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.856765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.857211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.858584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.861894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.885393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1052/2133] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:38.885587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c8dd6-a5f8-4d16-b2fa-bdeb50ee681d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.885690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:38.888613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.889476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.889878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.890277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.892674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:38.896985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389bb82-a771-49e8-8659-cf3268d9c85d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:38.902788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:38.903666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.903954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.904237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] PATH_INFO: `/v3/projects/fe5c3c26811f4e87b873916872543522/users/c3f1719ccd134173adc3d41eafc7dd03/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.906266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c3c26811f4e87b873916872543522, user_id=c3f1719ccd134173adc3d41eafc7dd03, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.909195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767e60b-f817-4334-ae81-b281068714f9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.920297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1053/2134] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:38 2022] PUT /identity/v3/projects/fe5c3c26811f4e87b873916872543522/users/c3f1719ccd134173adc3d41eafc7dd03/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:38.932705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f743f-8661-475a-8874-dba0eec63347 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:38.933241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f743f-8661-475a-8874-dba0eec63347 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:38.933512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f743f-8661-475a-8874-dba0eec63347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:38.952528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:38.962719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:38.964311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:38.964762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-554f743f-8661-475a-8874-dba0eec63347 None None] MFA Rules not processed for user `c3f1719ccd134173adc3d41eafc7dd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:38.966019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-554f743f-8661-475a-8874-dba0eec63347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:38.974596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1082/2135] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:39.052370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1054/2136] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:39.064541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:39.066711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:39.078895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bf4d7-a5ab-469b-a6e1-2c2afd15c75d service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:39.143772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:39.144384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:39.144866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:39.145309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:39.217006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:39.218496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:39.228878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1083/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:39.306521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:41.934606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f743f-8661-475a-8874-dba0eec63347 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:41.946131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f743f-8661-475a-8874-dba0eec63347 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:41.952183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:41.952835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:41.953122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:41.953439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:42.025196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:42.027445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:42.040431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1055/2138] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:44.498696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:44.537839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431ceb6-f44f-4312-970e-b9ace092b037 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:44.543040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:44.547344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:44.547344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:44.547344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:44.561117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:44.563331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f729be-85dc-4549-8409-d19496a1ffc9 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:44.577311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1084/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:44.800156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:44.805036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:44.906232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:44.938262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ede71f-82a7-475a-8b20-75ecc6202ae2 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:44.949784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:44.957066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:44.962795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:44.962795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] PATH_INFO: `/v3/users/cf8009de53a2414b86eae128241a1be2` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:44.962795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8009de53a2414b86eae128241a1be2)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:44.962795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:45.193327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8009de53a2414b86eae128241a1be2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:45.207925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8009de53a2414b86eae128241a1be2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:45.306410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8009de53a2414b86eae128241a1be2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:45.307077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8009de53a2414b86eae128241a1be2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:45.307693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8009de53a2414b86eae128241a1be2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:45.321782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1056/2140] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:44 2022] DELETE /identity/v3/users/cf8009de53a2414b86eae128241a1be2 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:45.326391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:45.326716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:45.327027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:45.349577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:45.350821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:45.426374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1085/2141] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:45.439065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:45.442472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:45.505767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e5cc5-a26c-4d2b-8d86-3cc9d2bd10e6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:45.509285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:45.509979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:45.510290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:45.510573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:45.520779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:45.522278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:45.532657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1057/2142] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:46.233831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:46.235322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:46.248410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a4642-fd14-46f8-838f-9c15201a6b01 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:46.256064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:46.256856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:46.257148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:46.257471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] PATH_INFO: `/v3/projects/4483daf6876c4f44a6cd6e281b35c16b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:46.259190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4483daf6876c4f44a6cd6e281b35c16b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:46.261017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:46.273897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:46.507679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4483daf6876c4f44a6cd6e281b35c16b', 'request_id': 'req-554eac2e-4ba2-4433-88df-9f57da278c14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:46.515982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4483daf6876c4f44a6cd6e281b35c16b', 'request_id': 'req-554eac2e-4ba2-4433-88df-9f57da278c14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:46.637904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4483daf6876c4f44a6cd6e281b35c16b', 'request_id': 'req-554eac2e-4ba2-4433-88df-9f57da278c14'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:46.654903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] The token cache is being invalidate because project 4483daf6876c4f44a6cd6e281b35c16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:46.655483 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:46.656551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:46.660347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1086/2143] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:46 2022] DELETE /identity/v3/projects/4483daf6876c4f44a6cd6e281b35c16b => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:46.665839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:46.677363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8734a9-9fc0-40ea-8d97-00dc93ae3162 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:46.743936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:46.744851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:46.745144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:46.745620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] PATH_INFO: `/v3/users/306bda7b6ae44f15a4d120e3e0d87d3b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:46.746950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] RBAC: Authorizing `identity:delete_user(user_id=306bda7b6ae44f15a4d120e3e0d87d3b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:46.748912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:46.760920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:47.182049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306bda7b6ae44f15a4d120e3e0d87d3b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:47.199720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306bda7b6ae44f15a4d120e3e0d87d3b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:47.676511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306bda7b6ae44f15a4d120e3e0d87d3b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:47.678034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306bda7b6ae44f15a4d120e3e0d87d3b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:47.679314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306bda7b6ae44f15a4d120e3e0d87d3b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:47.706705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1058/2144] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:46 2022] DELETE /identity/v3/users/306bda7b6ae44f15a4d120e3e0d87d3b => generated 0 bytes in 1038 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:48.320392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:48.321915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:48.344135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554eac2e-4ba2-4433-88df-9f57da278c14 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:48.427450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:48.428561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:48.429027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:48.429306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] PATH_INFO: `/v3/projects/131464c100354ca7974dff994292c25c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:48.430628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=131464c100354ca7974dff994292c25c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:48.432798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:48.634792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131464c100354ca7974dff994292c25c', 'request_id': 'req-92232be6-a724-4dbd-a9da-66c1baf22ba4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:48.643429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131464c100354ca7974dff994292c25c', 'request_id': 'req-92232be6-a724-4dbd-a9da-66c1baf22ba4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:48.785886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131464c100354ca7974dff994292c25c', 'request_id': 'req-92232be6-a724-4dbd-a9da-66c1baf22ba4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:48.798570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] The token cache is being invalidate because project 131464c100354ca7974dff994292c25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:48.798870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:48.800183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:48.801884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1087/2145] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:48 2022] DELETE /identity/v3/projects/131464c100354ca7974dff994292c25c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:48.806676 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:48.809710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:48.836267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370c95-ee4d-42e8-95d3-059779ef784a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:48.937265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:48.944268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:48.944268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:48.944268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] PATH_INFO: `/v3/users/53dcf6cdab86453a9d655261929e4767` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:48.946429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=53dcf6cdab86453a9d655261929e4767)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:48.950771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:49.428227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53dcf6cdab86453a9d655261929e4767'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:49.440993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53dcf6cdab86453a9d655261929e4767'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:49.678432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53dcf6cdab86453a9d655261929e4767'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:49.679062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53dcf6cdab86453a9d655261929e4767'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:49.679680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-515f1f7f-51ae-403c-a95f-85d4781fd8dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53dcf6cdab86453a9d655261929e4767'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:49.693264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1059/2146] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:48 2022] DELETE /identity/v3/users/53dcf6cdab86453a9d655261929e4767 => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:50.269770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:50.269770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:50.296651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232be6-a724-4dbd-a9da-66c1baf22ba4 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:50.337750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:50.395768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:50.397884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:50.398193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:50.398474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] PATH_INFO: `/v3/projects/188df723b4464985a8046aec80f62775` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:50.399867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] RBAC: Authorizing `identity:delete_project(project_id=188df723b4464985a8046aec80f62775)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:50.402187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:50.684490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '188df723b4464985a8046aec80f62775', 'request_id': 'req-87e99564-d5eb-4210-91bb-8d2560df660d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:50.693518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '188df723b4464985a8046aec80f62775', 'request_id': 'req-87e99564-d5eb-4210-91bb-8d2560df660d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:50.806423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '188df723b4464985a8046aec80f62775', 'request_id': 'req-87e99564-d5eb-4210-91bb-8d2560df660d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:50.823744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] The token cache is being invalidate because project 188df723b4464985a8046aec80f62775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:50.824289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:50.825231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-87e99564-d5eb-4210-91bb-8d2560df660d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:50.827919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1088/2147] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:50 2022] DELETE /identity/v3/projects/188df723b4464985a8046aec80f62775 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:50.832979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:50.833687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:50.833687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:50.850429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:50.860501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:50.860501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:50.956335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1060/2148] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:50.963838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:50.964137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:50.964422 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.005272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:51.006858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:51.021712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1089/2149] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:51.022200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:51.034745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263db7-a425-45c3-9c97-d86e316a6e10 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:51.046824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:51.047844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:51.048352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:51.048776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.050952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:51.051276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:51.053405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323de072-8466-4a59-b9d0-035e15394f7b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:51.087665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1061/2150] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:30:51.089060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:51.109176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0559-41ec-4c3a-a1d6-8af5fc625a11 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:51.117313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:51.118688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:51.119001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:51.119314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.121937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:51.124631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:51.151665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:51.151975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:51.152239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.203849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:51.205149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:51.316652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1062/2151] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:51.381094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:51.382871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:51.466713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c384dd-178a-447c-9f51-5ddea17b374f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:51.473036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:51.473787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:51.474071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:51.474350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.483702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:51.485614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:51.496209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1063/2152] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:30:51.672891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1090/2153] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:51 2022] POST /identity/v3/projects => generated 371 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:51.692065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:51.704032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa969db-420c-48fd-85be-5c189a6ff657 service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:51.821046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:51.821995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:51.822421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:51.822760 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:51.823600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:51.825874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:52.008365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1064/2154] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:51 2022] POST /identity/v3/users => generated 286 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:52.010019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:52.029170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc3e44-cf59-46d5-a4ba-eea7dbd28ac1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:52.039223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:52.040462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.042404 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.044834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.046263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:52.050809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:52.081416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1091/2155] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:52.083785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:52.107587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e716e0-ba9a-4a78-87aa-e8dd71fc8645 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:52.118137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:52.118929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.119381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.119799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.120640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:52.122500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c9e11-dc00-44fb-ab65-673815e4e376 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:52.139779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1065/2156] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:52.141839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:52.155023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4cb47-794a-4e91-afa1-d0cc0c1ff050 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:52.160382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:52.161370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.161751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.162113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] PATH_INFO: `/v3/projects/7ff8640432d844b5b4d1263c571f9048/users/8bf8e8f25ddd4a988b182e0716902135/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.178855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff8640432d844b5b4d1263c571f9048, user_id=8bf8e8f25ddd4a988b182e0716902135, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:52.181964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b2011-5a4f-4d5d-a6f5-ea7b4c6ded1b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:52.303562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1092/2157] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:52 2022] PUT /identity/v3/projects/7ff8640432d844b5b4d1263c571f9048/users/8bf8e8f25ddd4a988b182e0716902135/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:52.319920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.320485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.321060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.400057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] MFA Rules not processed for user `8bf8e8f25ddd4a988b182e0716902135`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:52.401590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:52.543640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3e5e-f4da-45c4-991f-a336c8a1e9f6 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.543974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3e5e-f4da-45c4-991f-a336c8a1e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.544259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3e5e-f4da-45c4-991f-a336c8a1e9f6 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.546109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1093/2158] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:30:52.570759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.570759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.570759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.630792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1066/2159] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:52.681327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] MFA Rules not processed for user `ce1f644a909746fe92bd0dee4e9ce635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:52.684261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:52.721734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:52.808129 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1094/2160] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:30:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:30:52.827873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:52.827873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:52.895258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f6da5-f71b-4b54-8ca3-bcfafc8e3536 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:52.905715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:52.907092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:52.912742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:52.913382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:52.928951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:52.936498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:52.960378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:52.979525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1067/2161] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:53.317310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:53.321256 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:53.421023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8f264-7f55-4075-a9b3-6225fa55aeea None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:53.427544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:53.428820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:53.429989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:53.430441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] PATH_INFO: `/v3/users/c5c44bb198b24d73b15f3b6bf1027aec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:53.434218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c44bb198b24d73b15f3b6bf1027aec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:53.445608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:53.836718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c44bb198b24d73b15f3b6bf1027aec'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:53.856309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c44bb198b24d73b15f3b6bf1027aec'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.045566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c44bb198b24d73b15f3b6bf1027aec'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.046421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c44bb198b24d73b15f3b6bf1027aec'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.095938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-0fdc3e5b-48b5-425c-83b7-0dbd380cdfb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c44bb198b24d73b15f3b6bf1027aec'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.116178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:54.117702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:54.148115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835d21-307d-441b-90a4-7b5cb128c901 service placement] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:54.153229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1095/2162] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:53 2022] DELETE /identity/v3/users/c5c44bb198b24d73b15f3b6bf1027aec => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:54.161482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:54.161482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:54.161482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:54.166006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:54.168696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:54.182227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:54.182520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] PATH_INFO: `/v3/users/d34c1c54534d4eaa8cc50425066b7618` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:54.185498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d34c1c54534d4eaa8cc50425066b7618)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:54.191731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:54.193663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:54.196531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:54.348008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1096/2163] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:54.368877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:54.369055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:54.484799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29cb80-68a6-4992-96c9-ab5a7a1d4f95 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:54.490013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:54.490727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:54.491009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:54.491314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:54.505536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:54.507004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:54.516618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd34c1c54534d4eaa8cc50425066b7618'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.519425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1097/2164] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:30:54.557793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:54.559412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd34c1c54534d4eaa8cc50425066b7618'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.713255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd34c1c54534d4eaa8cc50425066b7618'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.717279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd34c1c54534d4eaa8cc50425066b7618'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.717279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd34c1c54534d4eaa8cc50425066b7618'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:54.741082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1068/2165] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:54 2022] DELETE /identity/v3/users/d34c1c54534d4eaa8cc50425066b7618 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:54.919416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:55.382673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:55.384600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:55.446684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283e1-1cf5-4f67-9ed9-58e631761d62 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:55.540080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:55.546683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:55.578813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:55.579729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:55.580021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:55.580356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] PATH_INFO: `/v3/projects/aac437473fae4632840f5089550fb84f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:55.582627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac437473fae4632840f5089550fb84f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:55.584783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:55.694800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688472a-987d-4736-a1d8-2cefbc90637a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:55.703251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:55.705233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:55.708380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:55.708380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:55.709885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:55.780025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac437473fae4632840f5089550fb84f', 'request_id': 'req-e29cbb63-19a0-4dbf-8226-3fc33d51b402'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:55.790145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac437473fae4632840f5089550fb84f', 'request_id': 'req-e29cbb63-19a0-4dbf-8226-3fc33d51b402'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:55.872421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:55.873945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:55.884728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac437473fae4632840f5089550fb84f', 'request_id': 'req-e29cbb63-19a0-4dbf-8226-3fc33d51b402'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:55.887190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1069/2166] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:30:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:30:55.888163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:55.890126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:55.898588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] The token cache is being invalidate because project aac437473fae4632840f5089550fb84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:55.898893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:55.899578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:55.901460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1098/2167] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:55 2022] DELETE /identity/v3/projects/aac437473fae4632840f5089550fb84f => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:55.905035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:55.906573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:55.916166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957bc88-01fc-4d34-bdc9-d0df4a5dc220 service glance] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:56.120340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cbb63-19a0-4dbf-8226-3fc33d51b402 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:56.136854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:56.137750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:56.138030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:56.138306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] PATH_INFO: `/v3/projects/1e34322f6aa14465afc11a3842161882` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:56.139557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e34322f6aa14465afc11a3842161882)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:56.141514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:56.153615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:56.154531 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:56.154816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:56.155086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] PATH_INFO: `/v3/users/ad66fd3c657c40fd8c45d630b42b797e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:56.160541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad66fd3c657c40fd8c45d630b42b797e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:56.164962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:56.942195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e34322f6aa14465afc11a3842161882', 'request_id': 'req-31c07a95-60fd-4715-a3dc-a61d107f6401'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.121203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e34322f6aa14465afc11a3842161882', 'request_id': 'req-31c07a95-60fd-4715-a3dc-a61d107f6401'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.130838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e34322f6aa14465afc11a3842161882', 'request_id': 'req-31c07a95-60fd-4715-a3dc-a61d107f6401'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.142897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] The token cache is being invalidate because project 1e34322f6aa14465afc11a3842161882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:57.143758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.145202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.147235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1070/2168] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:55 2022] DELETE /identity/v3/projects/1e34322f6aa14465afc11a3842161882 => generated 0 bytes in 1259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:57.151261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:57.151415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad66fd3c657c40fd8c45d630b42b797e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.152906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad66fd3c657c40fd8c45d630b42b797e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.153639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:57.230452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c07a95-60fd-4715-a3dc-a61d107f6401 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:57.235150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:57.236037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:57.236329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:57.236607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] PATH_INFO: `/v3/users/8af62d95a1794c0bbf7855277236f3ea` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:57.238057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af62d95a1794c0bbf7855277236f3ea)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:57.240271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:57.361511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad66fd3c657c40fd8c45d630b42b797e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.364884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad66fd3c657c40fd8c45d630b42b797e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.397271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ccd0f431-a948-4d23-a0bb-68c724dfdda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad66fd3c657c40fd8c45d630b42b797e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.420987 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1099/2169] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:55 2022] DELETE /identity/v3/users/ad66fd3c657c40fd8c45d630b42b797e => generated 0 bytes in 1516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:57.422385 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:57.422719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:57.423023 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:57.447727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] MFA Rules not processed for user `cf3e0adc32814585b57bf9f884840fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:57.448994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:57.542590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1100/2170] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:57.608487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af62d95a1794c0bbf7855277236f3ea'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.641868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af62d95a1794c0bbf7855277236f3ea'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.713553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af62d95a1794c0bbf7855277236f3ea'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.714282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af62d95a1794c0bbf7855277236f3ea'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.715047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af62d95a1794c0bbf7855277236f3ea'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:57.746208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1071/2171] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:57 2022] DELETE /identity/v3/users/8af62d95a1794c0bbf7855277236f3ea => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:57.899266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:57.900858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:58.095890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ef32b-e0fb-4b46-9706-d4e568033996 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:58.230577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:58.233499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:58.233821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:58.234106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] PATH_INFO: `/v3/projects/b27dc5abdef34480942ca19891961563` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:58.241259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27dc5abdef34480942ca19891961563)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:58.248681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:58.470800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27dc5abdef34480942ca19891961563', 'request_id': 'req-219b2b91-3e5f-417a-8235-edf9dde111f0'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:58.505259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27dc5abdef34480942ca19891961563', 'request_id': 'req-219b2b91-3e5f-417a-8235-edf9dde111f0'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:58.622499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27dc5abdef34480942ca19891961563', 'request_id': 'req-219b2b91-3e5f-417a-8235-edf9dde111f0'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:58.651562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:58.653424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] The token cache is being invalidate because project b27dc5abdef34480942ca19891961563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:58.660513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:58.660513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-219b2b91-3e5f-417a-8235-edf9dde111f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:58.660894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:58.662075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:58.667509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1101/2172] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:57 2022] DELETE /identity/v3/projects/b27dc5abdef34480942ca19891961563 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:30:58.672785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2de63-ddae-44fa-91cc-67c678247266 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:58.674726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2de63-ddae-44fa-91cc-67c678247266 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:58.674956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2de63-ddae-44fa-91cc-67c678247266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:58.716279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654bd4e-c1cc-4af2-beb2-1662a26bdbd3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:58.724960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9fd2de63-ddae-44fa-91cc-67c678247266 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:58.730740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9fd2de63-ddae-44fa-91cc-67c678247266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:58.819951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:30:58.856653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:58.860901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:58.860901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:58.860901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] PATH_INFO: `/v3/projects/45c3b1f2cea34ce182bbfe61a7c450d7` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:58.860901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c3b1f2cea34ce182bbfe61a7c450d7)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:58.864700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:58.881921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1102/2173] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:58.893441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:58.893441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:58.893441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:58.962203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:30:58.963471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:58.990540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1103/2174] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:30:58.997491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.009783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba46be1-5ec4-4c8c-b857-dde2a9f66e1e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.020992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.025433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.025949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.026393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.028802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:30:59.029072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.032360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.054544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1104/2175] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:30:59.059947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.078127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa3ad-19ea-4c9b-ae0d-3a26443d693b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.086386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.087442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.087939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.090397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.091443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.094314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.130198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c3b1f2cea34ce182bbfe61a7c450d7', 'request_id': 'req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.272280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1105/2176] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:30:59 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:59.273000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.295042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c3b1f2cea34ce182bbfe61a7c450d7', 'request_id': 'req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.310936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c3b1f2cea34ce182bbfe61a7c450d7', 'request_id': 'req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.338866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] The token cache is being invalidate because project 45c3b1f2cea34ce182bbfe61a7c450d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:30:59.339798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.343264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.349832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1072/2177] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:30:58 2022] DELETE /identity/v3/projects/45c3b1f2cea34ce182bbfe61a7c450d7 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:30:59.354069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.355728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:59.397718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9aab2-ae70-4080-a0d4-a8e530e99db5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.404769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.406310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.406892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.407318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.408169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.410510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.458359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f2c0d-b76c-4dc2-8d1e-508e0bd5abfc admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.469386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.470791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.471685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.472360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] PATH_INFO: `/v3/users/c3f1719ccd134173adc3d41eafc7dd03` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.474706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f1719ccd134173adc3d41eafc7dd03)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.477472 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.630343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1106/2178] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:30:59 2022] POST /identity/v3/users => generated 292 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:30:59.639596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.649242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:30:59.681549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf3b66-a8cd-453d-a53f-debf3a167723 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.702342 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.703025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.703313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.703591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.704436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.706179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.738685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1107/2179] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:59.748150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.775368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875f375-8643-4dd8-b9e7-6affda18b17a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.795184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f1719ccd134173adc3d41eafc7dd03'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.796809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.797535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.798168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.798168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.798831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.801587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.825211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1108/2180] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:30:59.826337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:30:59.831876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f1719ccd134173adc3d41eafc7dd03'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.847485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54606cac-42c6-4788-826b-3df589fba2ef admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:30:59.866511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:30:59.867452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:30:59.867733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:30:59.868013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] PATH_INFO: `/v3/projects/bf8f192a31204d97b3b2e8895856a459/users/650592005e10426b95a76ebd97b07455/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:30:59.893930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8f192a31204d97b3b2e8895856a459, user_id=650592005e10426b95a76ebd97b07455, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:30:59.897591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:30:59.933236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f1719ccd134173adc3d41eafc7dd03'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.933236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f1719ccd134173adc3d41eafc7dd03'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.938335 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed0c9b11-b684-4453-9c95-cf5196b43fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f1719ccd134173adc3d41eafc7dd03'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:30:59.977392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1073/2181] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:30:59 2022] DELETE /identity/v3/users/c3f1719ccd134173adc3d41eafc7dd03 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:00.092062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1109/2182] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:30:59 2022] PUT /identity/v3/projects/bf8f192a31204d97b3b2e8895856a459/users/650592005e10426b95a76ebd97b07455/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:00.100569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:00.101113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:00.101558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:00.136809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:00.138692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:00.234150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1074/2183] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:00.243195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:00.245148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:00.414829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583ce03-778a-45d5-937d-863a97d08b7f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:00.422753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:00.425221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:00.425663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:00.426068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:00.443258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:00.446328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aba60e-2ec3-4e36-8d97-15e25daf52f9 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:00.463186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1110/2184] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:00.866956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:00.870065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:00.900097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0652a-f0ab-4144-8417-68f98e32b701 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:00.932678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:00.939249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:00.939249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:00.939249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] PATH_INFO: `/v3/projects/fe5c3c26811f4e87b873916872543522` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:00.950503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5c3c26811f4e87b873916872543522)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:00.954946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:01.401731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5c3c26811f4e87b873916872543522', 'request_id': 'req-7ab911c5-6380-454e-b0d3-372e905da626'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:01.589938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5c3c26811f4e87b873916872543522', 'request_id': 'req-7ab911c5-6380-454e-b0d3-372e905da626'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:01.600888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5c3c26811f4e87b873916872543522', 'request_id': 'req-7ab911c5-6380-454e-b0d3-372e905da626'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:01.623128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] The token cache is being invalidate because project fe5c3c26811f4e87b873916872543522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:31:01.623519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:01.624476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-7ab911c5-6380-454e-b0d3-372e905da626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:01.641929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1075/2185] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:31:00 2022] DELETE /identity/v3/projects/fe5c3c26811f4e87b873916872543522 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:01.642430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:01.642910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:01.644594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:01.695010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:01.705538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:01.812867 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1111/2186] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:01.827673 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:01.829966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:01.829966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:01.898933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:01.904337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:01.912297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:01.931816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1076/2187] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:01.934241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:01.947313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641b66-694a-4781-b92a-99cd43d1923a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:01.955550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:01.956605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:01.959117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:01.959646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:01.960625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:31:01.961126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:01.965079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:01.988269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1112/2188] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:31:01.994963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:02.007888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f9dfb-e9f0-44fc-ba44-564b04c4c354 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:02.022450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:02.023523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:02.024225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:02.025156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:02.027589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:02.033680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:02.947989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:02.956209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1077/2189] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:01 2022] POST /identity/v3/projects => generated 371 bytes in 963 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:02.961207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:02.972269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925e485-11ff-4d75-8524-9a3ee9473441 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:03.078321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:03.079033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:03.083103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:03.084099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:03.084874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:03.086919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:03.302576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1113/2190] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:02 2022] POST /identity/v3/users => generated 286 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:03.306838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:03.321532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5f65d-ad89-4fbd-b9b0-11dec358a338 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:03.337356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:03.339024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:03.340196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:03.340749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:03.347499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:03.351186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:03.374840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1078/2191] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:03.379971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:03.394131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337590d-dd4d-4d54-b439-655b49631883 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:03.404233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:03.405305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:03.406251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:03.407223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:03.408288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:03.412369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4a043-855b-4727-934e-c08f8bff98d8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:03.442157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1114/2192] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:03.442749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:03.485868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35285ff-e141-4e14-a518-ff91d1976b89 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:03.506793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:03.507701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:03.508227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:03.508668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] PATH_INFO: `/v3/projects/e1194991899d4dd5ac974608165516b8/users/97c3e6bd775848f39176015d42749fa2/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:03.556646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1194991899d4dd5ac974608165516b8, user_id=97c3e6bd775848f39176015d42749fa2, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:03.562207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:03.722711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1079/2193] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:03 2022] PUT /identity/v3/projects/e1194991899d4dd5ac974608165516b8/users/97c3e6bd775848f39176015d42749fa2/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:03.729329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067d76-9977-4df0-8c90-797916495ebe None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:03.729859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067d76-9977-4df0-8c90-797916495ebe None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:03.730297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7067d76-9977-4df0-8c90-797916495ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:03.784040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c7067d76-9977-4df0-8c90-797916495ebe None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:03.786582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c7067d76-9977-4df0-8c90-797916495ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:03.960141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1115/2194] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:03.992633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:04.003848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:04.296946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e8928-46f6-44d0-bc8b-ccbd68f5c3b8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:04.317577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:04.322534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:04.323412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:04.324658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:04.358207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:04.360841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:04.395603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1080/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:06.745677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:07.252494 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:07.795344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47537-b486-4d11-81a5-d4e9403fd2cf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:07.796476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47537-b486-4d11-81a5-d4e9403fd2cf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:07.796838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a47537-b486-4d11-81a5-d4e9403fd2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:07.914545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-14a47537-b486-4d11-81a5-d4e9403fd2cf None None] MFA Rules not processed for user `97c3e6bd775848f39176015d42749fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:07.915837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-14a47537-b486-4d11-81a5-d4e9403fd2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:08.169539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1116/2196] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:08.188849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:08.188849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:08.440606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e75a45-b579-410d-8ead-8f8b99057cdd service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:08.448782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:08.449956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:08.450426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:08.450892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:08.469377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:08.480097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:08.510956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1081/2197] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:09.374995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:09.502123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:11.120955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617b0a-26ad-47b9-b96f-b20889ac6403 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:11.121600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617b0a-26ad-47b9-b96f-b20889ac6403 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:11.121757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a617b0a-26ad-47b9-b96f-b20889ac6403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:11.147130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-0a617b0a-26ad-47b9-b96f-b20889ac6403 None None] MFA Rules not processed for user `97c3e6bd775848f39176015d42749fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:11.148590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0a617b0a-26ad-47b9-b96f-b20889ac6403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:11.163126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1117/2198] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:13.080069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:13.086916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:13.177995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2ab1a-bc2b-4dad-8708-dc997e205778 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:13.195165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:13.199225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:13.204658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:13.207876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] PATH_INFO: `/v3/users/89ba90ccf07f492fa121f0bb916c2870` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:13.212238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ba90ccf07f492fa121f0bb916c2870)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:13.216872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:13.331246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:13.570579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ba90ccf07f492fa121f0bb916c2870'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:13.609890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ba90ccf07f492fa121f0bb916c2870'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:13.730922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ba90ccf07f492fa121f0bb916c2870'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:13.731859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ba90ccf07f492fa121f0bb916c2870'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:13.732784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ba90ccf07f492fa121f0bb916c2870'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:13.756713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1082/2199] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:31:13 2022] DELETE /identity/v3/users/89ba90ccf07f492fa121f0bb916c2870 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:13.760733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09a560-58c2-49bf-a3e5-349ece6bac83 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:13.761030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09a560-58c2-49bf-a3e5-349ece6bac83 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:13.761312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09a560-58c2-49bf-a3e5-349ece6bac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:13.783201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5e09a560-58c2-49bf-a3e5-349ece6bac83 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:13.791655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5e09a560-58c2-49bf-a3e5-349ece6bac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:13.929394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1118/2200] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:13.941711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:13.950994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:14.039624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d3cb-58c4-48a4-a5c9-3220ebef5e2a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:14.044711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:14.045427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:14.046156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:14.046156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:14.063072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:14.064809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:14.077715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1083/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:14.677293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:15.173476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.175297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.177667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.195128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.196649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.218916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905df3a-d884-4ea5-a1bb-297a13e36148 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.236329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] MFA Rules not processed for user `650592005e10426b95a76ebd97b07455`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:15.237974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.247306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.249690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.251829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.252349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] PATH_INFO: `/v3/projects/7bbf24de48a64da48640a0d42ff0f2ca` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.254871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bbf24de48a64da48640a0d42ff0f2ca)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.258205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.324198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1119/2202] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:15.329042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.330779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.345187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d03df-1acb-4d1d-8aeb-42210f083672 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.351200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.351884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.352172 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.352449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.353143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.355143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.442587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bbf24de48a64da48640a0d42ff0f2ca', 'request_id': 'req-50da1011-5f3c-479c-aeda-343e18b3bbd2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:15.554146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1120/2203] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:31:15.558354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.583070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bbf24de48a64da48640a0d42ff0f2ca', 'request_id': 'req-50da1011-5f3c-479c-aeda-343e18b3bbd2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:15.596212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bbf24de48a64da48640a0d42ff0f2ca', 'request_id': 'req-50da1011-5f3c-479c-aeda-343e18b3bbd2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:15.612207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] The token cache is being invalidate because project 7bbf24de48a64da48640a0d42ff0f2ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:31:15.612724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:15.615204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-50da1011-5f3c-479c-aeda-343e18b3bbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:15.621076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1084/2204] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:31:15 2022] DELETE /identity/v3/projects/7bbf24de48a64da48640a0d42ff0f2ca => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:15.628269 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b8dd-aff8-47ea-b608-fe08673d2816 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.636248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b8dd-aff8-47ea-b608-fe08673d2816 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.636248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b8dd-aff8-47ea-b608-fe08673d2816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.642944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11419-9f0e-44de-90c3-24801e99e62b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.649187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.649955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.650242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.650695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.651453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.653522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.667683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d457b8dd-aff8-47ea-b608-fe08673d2816 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:15.669114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d457b8dd-aff8-47ea-b608-fe08673d2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.687178 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1085/2205] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:15.696502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.696927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.698944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.717207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:15.718468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.728616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1086/2206] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:15.732940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.745697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109d68-1909-4563-8172-7e95e6ee7d54 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.751291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.751981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.752277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.752564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.753356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:31:15.753901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.755539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.777142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1121/2207] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:15.781174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1087/2208] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:31:15.782083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.786462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.789798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:15.800566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0f6ae-4d28-4b8e-bd1f-bdcce25e27af admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.807737 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.809214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.810159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.810479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.810614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153f37f-5d85-46e2-a79e-8badb770814e admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.811647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.814053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.835401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.836230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.837082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.837466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.838374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.840348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.873638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1122/2209] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:15.877834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:15.894147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c54ce4-414b-4206-9006-8749868a92e9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:15.926642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:15.927797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:15.928563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:15.929025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] PATH_INFO: `/v3/projects/f51f25652579464588c765d769165de8/users/7c749c3b281142fcb674df9193d45f07/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:15.959343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51f25652579464588c765d769165de8, user_id=7c749c3b281142fcb674df9193d45f07, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:15.962743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:15.986487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1088/2210] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/projects => generated 391 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:15.991163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.119139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbef4d-15ec-4388-a8e0-202aa00b031e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.125474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.134273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.134273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.134273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.134273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.134668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1123/2211] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:15 2022] PUT /identity/v3/projects/f51f25652579464588c765d769165de8/users/7c749c3b281142fcb674df9193d45f07/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:16.134753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.136049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.224920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc4228-002f-428f-870b-42860d12a5be admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.231142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.231877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.232169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.232458 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.233190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.235197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.255319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1124/2212] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:16.256878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.271598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1ec40-8479-40ee-a0ce-9110d12fa775 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.277151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.278363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.278844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.279344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] PATH_INFO: `/v3/projects/f51f25652579464588c765d769165de8/users/7c749c3b281142fcb674df9193d45f07/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.281737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51f25652579464588c765d769165de8, user_id=7c749c3b281142fcb674df9193d45f07, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.285789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.298102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1089/2213] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:15 2022] POST /identity/v3/users => generated 296 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:31:16.303024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1125/2214] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:16 2022] PUT /identity/v3/projects/f51f25652579464588c765d769165de8/users/7c749c3b281142fcb674df9193d45f07/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:16.315174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.413377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5f2c7-6f7c-4819-a9dd-b0dcaef42140 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.418793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.419775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.420305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.420681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.421772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.423393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.436675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1126/2215] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:16.440728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.451423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d7e71-1b3d-45de-bb8c-f0729363a749 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.457320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.458668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.459110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.459540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.463202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.463202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aae5ca-76fc-4eb1-928b-6f41c4c89ff4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.474683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1090/2216] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:31:16.478636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.492631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cce6c-cd97-4318-9025-40cd35568150 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.500460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.500460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.500852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.501201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] PATH_INFO: `/v3/projects/0b6f8f1a8b01408e9eab4c8d76d77f0a/users/1fb40c31ef2a4a81a0a3f29b245f9a2c/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.531466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6f8f1a8b01408e9eab4c8d76d77f0a, user_id=1fb40c31ef2a4a81a0a3f29b245f9a2c, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.531677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:16.634775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1127/2217] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:16 2022] PUT /identity/v3/projects/0b6f8f1a8b01408e9eab4c8d76d77f0a/users/1fb40c31ef2a4a81a0a3f29b245f9a2c/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:31:16.645766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a355-98ff-4584-9519-3ab73e038853 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.646752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a355-98ff-4584-9519-3ab73e038853 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.647444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a355-98ff-4584-9519-3ab73e038853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.689380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2d10a355-98ff-4584-9519-3ab73e038853 None None] MFA Rules not processed for user `1fb40c31ef2a4a81a0a3f29b245f9a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:16.690753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2d10a355-98ff-4584-9519-3ab73e038853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:16.767479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:16.779169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1091/2218] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:16.791650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:16.793293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:16.952509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d379-387e-4365-a82b-432ec52fe0f7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:16.960595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:16.961264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:16.961615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:16.961889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:16.985292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:16.998212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341310b-b20f-49b3-9d8f-2a0df0240766 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:17.007215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1128/2219] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:17.821206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7f7a0-80e4-4f1b-a4e8-7beee1ba184a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:17.821906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7f7a0-80e4-4f1b-a4e8-7beee1ba184a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:17.821906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7f7a0-80e4-4f1b-a4e8-7beee1ba184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:17.845488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-57b7f7a0-80e4-4f1b-a4e8-7beee1ba184a None None] MFA Rules not processed for user `56799d191ffe4193918fa0221c27c0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:17.856012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-57b7f7a0-80e4-4f1b-a4e8-7beee1ba184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:17.954501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1092/2220] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:19.396700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:19.762610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:23.146605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:23.147668 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:23.148095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:23.155117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1129/2221] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:31:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:31:23.156583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276015-eb7a-40e4-b523-e38b94cdf9ec None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:23.157018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276015-eb7a-40e4-b523-e38b94cdf9ec None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:23.157181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb276015-eb7a-40e4-b523-e38b94cdf9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:23.236193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-bb276015-eb7a-40e4-b523-e38b94cdf9ec None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:23.238399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-bb276015-eb7a-40e4-b523-e38b94cdf9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:23.298003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:23.361418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1093/2222] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:31:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:23.368965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:23.372113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:23.476106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15709ec3-3cef-4c5f-8c53-79325f0af6be None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:23.480176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:23.480439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:23.480727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:23.481007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:23.492831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:23.494330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d238f-b4d3-4419-92fb-be71a2ea4ed6 service glance] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:23.506242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1130/2223] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:23.836265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:23.836827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:23.837403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:23.841050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1094/2224] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:31:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:31:23.844231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2c45f-f319-4a16-b879-3b1010a29469 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:23.844537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2c45f-f319-4a16-b879-3b1010a29469 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:23.844828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2c45f-f319-4a16-b879-3b1010a29469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:23.871054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fec2c45f-f319-4a16-b879-3b1010a29469 None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:23.872321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fec2c45f-f319-4a16-b879-3b1010a29469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:23.971315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:23.986562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1131/2225] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:31:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:24.004812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:24.004812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:24.029795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cbed0-ecbd-4a75-be4f-554c35a308e2 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:24.039297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:24.040356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:24.040550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:24.040951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:24.064319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:24.066516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c856e3c-4cb9-4496-b20d-3a503177b91e service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:24.087053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1095/2226] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:24.240585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04176-9290-49af-a727-f16edf6e5913 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:24.240585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04176-9290-49af-a727-f16edf6e5913 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:24.240585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04176-9290-49af-a727-f16edf6e5913 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:24.240585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1132/2227] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:31:24.241252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ed68-f222-433f-bb06-61559bc7109a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:24.241252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ed68-f222-433f-bb06-61559bc7109a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:24.241252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ed68-f222-433f-bb06-61559bc7109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:24.263337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f831ed68-f222-433f-bb06-61559bc7109a None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:24.264587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f831ed68-f222-433f-bb06-61559bc7109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:24.277223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1096/2228] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:31:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:24.296095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec04176-9290-49af-a727-f16edf6e5913 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:24.320337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec04176-9290-49af-a727-f16edf6e5913 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:24.330567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:24.331392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:24.331679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:24.331961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:24.347029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:24.349809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:24.374998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1133/2229] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:26.526526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3701-29cc-489d-a8f4-dc077e44a839 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:26.528689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3701-29cc-489d-a8f4-dc077e44a839 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:26.529133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c3701-29cc-489d-a8f4-dc077e44a839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:26.575123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-534c3701-29cc-489d-a8f4-dc077e44a839 None None] MFA Rules not processed for user `7c749c3b281142fcb674df9193d45f07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:26.576744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-534c3701-29cc-489d-a8f4-dc077e44a839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:26.685158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1097/2230] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:26.693056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:26.706789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286bf9f-461b-486a-9836-18a1346c596f service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:26.712070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:26.713691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:26.714279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:26.714839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:26.720089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:26.799833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:26.801418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:26.820468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1134/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:31:27.143476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:27.144687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:27.145709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:27.205304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:27.206565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:27.300811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1098/2232] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:31:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:27.318726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:27.320367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:27.443095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f10a-6596-4088-88a7-b44ca00b0783 service glance] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:27.450587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:27.451457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:27.451752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:27.452156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:27.462645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:27.464254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52518-fbcd-4c02-8768-dbfcfbdf0b82 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:27.477974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1135/2233] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:31:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:31:37.296781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:37.470928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=77 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:42.225859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:42.227349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:42.300061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80a372-97aa-46d8-99d7-9f7821ba62d7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:42.307023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:42.307961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:42.308271 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:42.308549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] PATH_INFO: `/v3/users/8bf8e8f25ddd4a988b182e0716902135` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:42.310047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf8e8f25ddd4a988b182e0716902135)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:42.312016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:42.555253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c96-3fd2-4879-a48a-bab6bad66975 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:42.555927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c96-3fd2-4879-a48a-bab6bad66975 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:42.556236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c96-3fd2-4879-a48a-bab6bad66975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:42.578764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-088c8c96-3fd2-4879-a48a-bab6bad66975 None None] MFA Rules not processed for user `650592005e10426b95a76ebd97b07455`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:42.580001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-088c8c96-3fd2-4879-a48a-bab6bad66975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:42.588533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1136/2234] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:42.706242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf8e8f25ddd4a988b182e0716902135'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:42.721276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf8e8f25ddd4a988b182e0716902135'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:42.857700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf8e8f25ddd4a988b182e0716902135'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:42.858805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf8e8f25ddd4a988b182e0716902135'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:42.870077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf8e8f25ddd4a988b182e0716902135'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:42.874340 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1099/2235] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:31:42 2022] DELETE /identity/v3/users/8bf8e8f25ddd4a988b182e0716902135 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:42.878859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:42.879220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:42.879589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:42.898616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:42.899849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:42.974263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1137/2236] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:42.984654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:42.986146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:43.061745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61d1e1-699c-4832-abcf-e7b4e74462a0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:43.065536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:43.066232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:43.066520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:43.066802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:43.077745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:43.079212 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:43.090418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1100/2237] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:43.519610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:43.524407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:43.539952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000700f-010c-4b89-9e59-47aad6344092 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:43.545242 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:43.546252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:43.546536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:43.546819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] PATH_INFO: `/v3/users/1fb40c31ef2a4a81a0a3f29b245f9a2c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:43.548216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb40c31ef2a4a81a0a3f29b245f9a2c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:43.550268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:43.659819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:43.662139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:43.675026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a183b-ee40-4822-91f2-4728aeee89aa service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:43.680506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:43.681396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:43.681725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:43.681961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] PATH_INFO: `/v3/projects/7ff8640432d844b5b4d1263c571f9048` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:43.683146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff8640432d844b5b4d1263c571f9048)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:43.685110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:43.704490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:43.818852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb40c31ef2a4a81a0a3f29b245f9a2c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.821826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb40c31ef2a4a81a0a3f29b245f9a2c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.836111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff8640432d844b5b4d1263c571f9048', 'request_id': 'req-84f42158-b9ff-4af7-a8dd-592166e68e4b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.949660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb40c31ef2a4a81a0a3f29b245f9a2c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.953264 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb40c31ef2a4a81a0a3f29b245f9a2c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.974596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-85f960ea-3de9-466f-b028-9fd7c8b94e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb40c31ef2a4a81a0a3f29b245f9a2c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:43.990120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1138/2238] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:31:43 2022] DELETE /identity/v3/users/1fb40c31ef2a4a81a0a3f29b245f9a2c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:43.994674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:43.994972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:43.995310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.014266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:44.016325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.049182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff8640432d844b5b4d1263c571f9048', 'request_id': 'req-84f42158-b9ff-4af7-a8dd-592166e68e4b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:44.056613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff8640432d844b5b4d1263c571f9048', 'request_id': 'req-84f42158-b9ff-4af7-a8dd-592166e68e4b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:44.070374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] The token cache is being invalidate because project 7ff8640432d844b5b4d1263c571f9048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:31:44.077605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:44.077605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-84f42158-b9ff-4af7-a8dd-592166e68e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:44.077605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1101/2239] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:31:43 2022] DELETE /identity/v3/projects/7ff8640432d844b5b4d1263c571f9048 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:31:44.079817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea619607-7b46-4a42-a85f-ed702e6c5020 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.080274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea619607-7b46-4a42-a85f-ed702e6c5020 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.080704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea619607-7b46-4a42-a85f-ed702e6c5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.083122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1139/2240] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:44.092935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.094625 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.102814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ea619607-7b46-4a42-a85f-ed702e6c5020 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:44.104052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ea619607-7b46-4a42-a85f-ed702e6c5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.106501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:44.114545 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1102/2241] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:44.123713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.124118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.124330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.168137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:44.169401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.182157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1103/2242] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:44.186431 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.204344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b96a6-2a2a-43d0-af4b-bbb6c2149775 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.205032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f62c9-d7ab-4ada-80b1-bd9dfed6a1a8 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.210136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.215447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.215447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.215447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.215725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.215725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.215725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.215725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.216006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:31:44.216213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.218347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131821b6-5f6f-42b4-827d-df687bcaa006 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.222481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.224557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.237562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1104/2243] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:31:44.237562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.237562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.237958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1140/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:31:44.238054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.239439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.254367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b684dc2-cb24-41ec-88c9-cdc402b43fcd service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.261017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.262767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.263268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.263727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.264824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.265573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:44.266830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.268554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.357885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1105/2245] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:44.371628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.372925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.400567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1141/2246] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:31:44.407045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.461833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4ada9-2c15-4777-96ea-fa8ffdd99214 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.510946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c397907-057a-4c2d-99e0-ea692d93a2c6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.510946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.510946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.510946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.511262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.511765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.513860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.538557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.539872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.540101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.540379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.625911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.628011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.638137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1106/2247] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:31:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:31:44.701932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1142/2248] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/users => generated 287 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:44.706329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.718118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac0f44-b11d-4271-a397-74180e27eeef service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.723229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.724089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.724903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.725461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.726215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.730405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.739159 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1107/2249] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:44.743872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.757929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300debe-494d-406d-86ca-9f460641d84f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.762750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.763509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.763801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.764083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.764902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.766564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.781815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1143/2250] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:44.783152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.795881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd01e1a-ae77-4283-87c2-5972bd63a586 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.800887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.801938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.802276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.802571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] PATH_INFO: `/v3/projects/6645a6417f5f4237b899b8d97ee1d7f3/users/2275ff79ad5043e7b7b0590cea10ff9f/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.827126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6645a6417f5f4237b899b8d97ee1d7f3, user_id=2275ff79ad5043e7b7b0590cea10ff9f, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.831806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d0c03-1d1b-482d-a197-462cd9269ea4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.850280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:44.853546 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:44.904186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d60bab-75a6-49b7-b278-7c3acb87ba86 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:44.917112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:44.917915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.918205 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.918496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] PATH_INFO: `/v3/projects/0b6f8f1a8b01408e9eab4c8d76d77f0a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:44.919996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6f8f1a8b01408e9eab4c8d76d77f0a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:44.922019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:44.979613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1108/2251] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:44 2022] PUT /identity/v3/projects/6645a6417f5f4237b899b8d97ee1d7f3/users/2275ff79ad5043e7b7b0590cea10ff9f/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:44.990436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:44.990791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:44.991071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.030080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] MFA Rules not processed for user `2275ff79ad5043e7b7b0590cea10ff9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:45.031307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.109663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6f8f1a8b01408e9eab4c8d76d77f0a', 'request_id': 'req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.115903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6f8f1a8b01408e9eab4c8d76d77f0a', 'request_id': 'req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.129037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1109/2252] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:45.141511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:45.142951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.211229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd500ac-e62b-4b4b-945e-b52139a29788 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:45.234921 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6f8f1a8b01408e9eab4c8d76d77f0a', 'request_id': 'req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.264634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] The token cache is being invalidate because project 0b6f8f1a8b01408e9eab4c8d76d77f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:31:45.265008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.266799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-851114cf-a7ed-4e6b-8fb8-bd040c4c340d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.273233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1144/2253] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:31:44 2022] DELETE /identity/v3/projects/0b6f8f1a8b01408e9eab4c8d76d77f0a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:31:45.294766 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.295323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.295759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.331145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:45.333279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.334246 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.335151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.338164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:45.339804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.340295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.457822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:45.460418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756833-81d0-4a59-9275-7eb8671b60cf service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:45.476140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1110/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:31:45.487979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1145/2255] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:45.497261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.497642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.497975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.510277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:45.511734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.528498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f1c1-a39f-4d25-85fb-8398e9a64e07 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:45.539409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:45.540581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:45.543089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:45.544356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.544868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.545291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] PATH_INFO: `/v3/users/2275ff79ad5043e7b7b0590cea10ff9f` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.548465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2275ff79ad5043e7b7b0590cea10ff9f)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:45.551689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:45.578445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1111/2256] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:45.586496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:45.610924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286cc04-94ac-45f6-954d-cb728b3d957e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:45.613008 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:45.613728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.614014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.614291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.615389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:31:45.615687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:45.619373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:45.639910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1112/2257] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:31:45.650622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:45.680907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9c6e-814a-4995-a989-d4a23c507c3e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:45.691264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:45.692377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.692765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.694976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.694976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:45.698430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:45.835838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1113/2258] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:45 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:31:45.845233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:45.890094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2275ff79ad5043e7b7b0590cea10ff9f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.891116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2275ff79ad5043e7b7b0590cea10ff9f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:45.946462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac4e6-6714-4f6f-90bf-f9d4428832be admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:45.950804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:45.951616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:45.955721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:45.955721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:45.955721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:45.955721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.029537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2275ff79ad5043e7b7b0590cea10ff9f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:46.030610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2275ff79ad5043e7b7b0590cea10ff9f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:46.044693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-70647c92-5d3e-4455-a6f1-88d835bf35a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2275ff79ad5043e7b7b0590cea10ff9f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:46.063785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1146/2259] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:31:45 2022] DELETE /identity/v3/users/2275ff79ad5043e7b7b0590cea10ff9f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:46.068678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1114/2260] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:45 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:46.069633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.069835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:46.070012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.070211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.072115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:46.086886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aedaf-2e7e-47fa-876f-868fb1138cff admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:46.099480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:46.101405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:46.155591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:46.173983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:46.174705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.174945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.175228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.175963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:46.178140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.189975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1115/2261] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:46.196872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:46.205843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bddf3a-eaf0-4b33-949e-353745afd533 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:46.207518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1147/2262] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:31:46.211705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:46.212394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.212678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.212956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.218624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:46.220445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:46.228288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:46.231374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.248772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1116/2263] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:46.252633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:46.287238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382d113-f40d-43cf-8319-f82eb492aa0d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:46.296127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:46.297179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.297516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.297786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] PATH_INFO: `/v3/projects/e57270a9020844e9baf044630fb81bcd/users/672b81726d1445518823f6e7c8887978/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.306558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa417984-5c8a-4d79-bf4e-e0de9322a16f None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:46.311706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:46.312728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.313682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.313988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.328235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:46.330271 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49d6fd-bb3d-4296-8b31-a645677c60a4 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.343754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57270a9020844e9baf044630fb81bcd, user_id=672b81726d1445518823f6e7c8887978, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:46.346713 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.348335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1148/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:46.525023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1117/2265] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:46 2022] PUT /identity/v3/projects/e57270a9020844e9baf044630fb81bcd/users/672b81726d1445518823f6e7c8887978/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:31:46.526446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.529384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.529384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.540675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:46.554265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:46.557047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:46.682449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1149/2266] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:46.692432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:46.765548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd83d7-f3e3-4530-9719-4fdf8701782f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:46.770380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:46.772799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:46.772799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:46.773071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:46.791889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:46.793880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41021f0-3650-42cb-8d79-191ec11c420d service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:46.821628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1118/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:31:47.101593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:47.106821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:47.126454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e54dcb-b764-4162-acef-5d3a4ee70e11 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:47.133958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:47.134920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:47.135204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:47.135633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] PATH_INFO: `/v3/projects/6645a6417f5f4237b899b8d97ee1d7f3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:47.137233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6645a6417f5f4237b899b8d97ee1d7f3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:47.139216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:47.309914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6645a6417f5f4237b899b8d97ee1d7f3', 'request_id': 'req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:47.326783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6645a6417f5f4237b899b8d97ee1d7f3', 'request_id': 'req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:47.415232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6645a6417f5f4237b899b8d97ee1d7f3', 'request_id': 'req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:47.430909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] The token cache is being invalidate because project 6645a6417f5f4237b899b8d97ee1d7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:31:47.431339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:47.433923 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:31:47.437260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1150/2268] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:31:47 2022] DELETE /identity/v3/projects/6645a6417f5f4237b899b8d97ee1d7f3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:47.767921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:47.970894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:49.067976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:55.539156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:55.540023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:55.540023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:55.616816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] MFA Rules not processed for user `672b81726d1445518823f6e7c8887978`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:55.621758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:55.665704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:55.668328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:55.687203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:55.718192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:55.757447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1119/2269] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:31:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:55.764898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:55.769884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:55.849469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f687f-02df-4a07-8324-b5c2ae6dbcd4 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:55.858778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:55.861683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:55.861683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:55.861683 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:55.863851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:55.908557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:55.910263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:55.923279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1151/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:31:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:31:55.990947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517961bc-90ee-4f3b-ae1a-7d37d5addb86 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:55.996686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:55.997387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:55.997939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:55.998326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:55.999194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.002188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.130326 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1120/2271] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:31:55 2022] POST /identity/v3/projects => generated 373 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:56.140449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:56.147873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657ac8b-6aae-4d2b-b4ea-09550b319165 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:56.214541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:56.215234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:56.215523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:56.215806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:56.216507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.218777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.365190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1152/2272] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:31:56 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:31:56.365916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:56.374343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492005-3454-47d0-a7e4-6088dd0f0a02 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:56.379094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:56.379779 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:56.380068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:56.380353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:56.381927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.383613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.395025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1121/2273] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:56.403471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:56.421806 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366eb3e-d27f-492f-bdec-2bac6b6bc9e3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:56.426924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:56.429218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:56.429622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:56.430102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] PATH_INFO: `/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15/users/dd7aaaa3c51c43c7ab27d122363c37aa/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:56.448383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40eb92d00fc34b498c8c0a44f27e2e15, user_id=dd7aaaa3c51c43c7ab27d122363c37aa, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.451873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.596510 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1153/2274] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:56 2022] PUT /identity/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15/users/dd7aaaa3c51c43c7ab27d122363c37aa/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:56.601465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:56.613035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7406a8d-36bb-41f1-85f3-10de181a338a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:56.694425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:56.695467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:56.695967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:56.696476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:56.697416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.699086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bec84a-6a14-4c47-bed6-a6a97e579614 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.713720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1122/2275] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:31:56.715581 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:56.727883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952311fd-6589-4dc9-be75-38b796bb9e8a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:56.736364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:56.736364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:56.736364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:56.736364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] PATH_INFO: `/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15/users/dd7aaaa3c51c43c7ab27d122363c37aa/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:56.736802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40eb92d00fc34b498c8c0a44f27e2e15, user_id=dd7aaaa3c51c43c7ab27d122363c37aa, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:56.739278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:56.751072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1154/2276] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:31:56 2022] PUT /identity/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15/users/dd7aaaa3c51c43c7ab27d122363c37aa/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:31:59.347689 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72d8-a9fe-4c82-9d19-c3566cfbdbbb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:59.348239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72d8-a9fe-4c82-9d19-c3566cfbdbbb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:59.348589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d72d8-a9fe-4c82-9d19-c3566cfbdbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:59.392047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-337d72d8-a9fe-4c82-9d19-c3566cfbdbbb None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:31:59.393912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-337d72d8-a9fe-4c82-9d19-c3566cfbdbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:59.474700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:59.506898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1123/2277] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:31:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:31:59.534109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:31:59.536313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:31:59.762163 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce262b7-b281-4a8a-8ff9-0ba982eb95dc admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:31:59.782461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:31:59.783393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:31:59.783904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:31:59.784411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:31:59.796856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:31:59.798948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d1f9c-1483-48b9-bf36-5f86c2ccb731 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:31:59.815152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:31:59.820567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1155/2278] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:31:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:32:01.355055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:01.355055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:01.355055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:01.398307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] MFA Rules not processed for user `97c3e6bd775848f39176015d42749fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:32:01.399759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:01.488193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1124/2279] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:32:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:32:06.567123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6a7b3-cf31-4b48-9410-327736b58cad None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:06.567665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6a7b3-cf31-4b48-9410-327736b58cad None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:06.567665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6a7b3-cf31-4b48-9410-327736b58cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:06.598539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c3f6a7b3-cf31-4b48-9410-327736b58cad None None] MFA Rules not processed for user `dd7aaaa3c51c43c7ab27d122363c37aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:32:06.599877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c3f6a7b3-cf31-4b48-9410-327736b58cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:06.699069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1156/2280] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:32:06.713612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:32:06.715150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:06.804403 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463733e-aabb-49e1-a675-dde7c283605e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:32:06.810851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:32:06.816489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:06.816489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:06.816489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:06.874911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:32:06.876629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90b2af-f6cb-4ad0-9ab9-911d7b518bc6 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:32:06.887328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1125/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:32:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:32:10.800402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:10.832460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:10.832460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:10.832460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] MFA Rules not processed for user `672b81726d1445518823f6e7c8887978`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:32:10.832460 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:10.842829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1157/2282] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:32:16.882577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=86 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:32:20.824931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=37 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:32:24.332866 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d0f2-25f9-4e0a-aebf-018a1689420c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:24.333362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d0f2-25f9-4e0a-aebf-018a1689420c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:24.333454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8d0f2-25f9-4e0a-aebf-018a1689420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:24.353769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-80d8d0f2-25f9-4e0a-aebf-018a1689420c None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:32:24.355054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-80d8d0f2-25f9-4e0a-aebf-018a1689420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:24.362442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1126/2283] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:32:24.375521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:32:24.388178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde243f-90b2-46b4-be51-fcf5c8bd1e32 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:32:24.393247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:32:24.393920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:24.394212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:24.394493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:24.403645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:32:24.405232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:32:24.413410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1158/2284] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:32:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:32:34.347473 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:32:34.410142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:32:47.987584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:47.988177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:47.988177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:48.008576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:32:48.010180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:48.018988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1127/2285] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:32:48.036943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:32:48.047937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab089-d83b-416c-a59c-ee288749f208 service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:32:48.053447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:32:48.054142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:48.067196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:48.067196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:48.067196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:32:48.067196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:32:48.077259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1159/2286] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:32:52.140841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:32:52.142912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:52.341065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc4966-474c-4b24-b118-38b26fb43436 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:32:52.346420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:32:52.347932 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:52.349261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:52.349850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:52.356437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:52.486314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:32:52.489162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27a35c-9bd1-426b-82b3-462b34fb84f4 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:32:52.514795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1128/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:32:57.350098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:32:57.351746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:32:57.379198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763606cd-2bce-44c1-a8e1-741e2ddc3888 service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:32:57.388341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:32:57.389638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:32:57.390137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:32:57.390580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:32:57.418515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:32:57.422566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a705421-2606-405d-9283-6b8fd24b8fe6 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:32:57.465596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1160/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:32:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:02.504325 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=53 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:07.456505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=12 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:08.393223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:08.393223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:08.393223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:08.420344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] MFA Rules not processed for user `cf3e0adc32814585b57bf9f884840fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:08.421665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:08.493588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1129/2289] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:11.400075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e9fa4-73e1-4581-bd01-cd73b4b2cc9f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:11.400650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e9fa4-73e1-4581-bd01-cd73b4b2cc9f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:11.400650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e9fa4-73e1-4581-bd01-cd73b4b2cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:11.419493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-758e9fa4-73e1-4581-bd01-cd73b4b2cc9f None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:11.420734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-758e9fa4-73e1-4581-bd01-cd73b4b2cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:11.428763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1161/2290] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:11.441741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:11.454899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5f9a4-f4bc-4d41-9d4b-220c8015b460 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:11.459867 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:11.460531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:11.460813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:11.461097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:11.470646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:11.472259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:11.481126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1130/2291] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:33:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:33:14.445197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:14.445723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:14.445825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:14.468229 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] MFA Rules not processed for user `56799d191ffe4193918fa0221c27c0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:14.469693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:14.533735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1162/2292] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:15.566862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:15.568443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:15.646655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384427-3a4c-4e0a-ade3-05eff2212cfe service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:15.651774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:15.652837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:15.653199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:15.653804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] PATH_INFO: `/v3/users/97c3e6bd775848f39176015d42749fa2` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:15.654823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c3e6bd775848f39176015d42749fa2)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:15.656852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:15.891656 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c3e6bd775848f39176015d42749fa2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:15.904412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c3e6bd775848f39176015d42749fa2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:15.981528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c3e6bd775848f39176015d42749fa2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:15.982221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c3e6bd775848f39176015d42749fa2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:15.982935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e13e17c3-182a-4f0d-96b9-b2fe4ae887b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c3e6bd775848f39176015d42749fa2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:15.998195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1131/2293] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:33:15 2022] DELETE /identity/v3/users/97c3e6bd775848f39176015d42749fa2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:33:16.284624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:16.304644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec329b-a327-4d0e-9a1c-851498dcfc21 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:16.369990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:16.370858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:16.371181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:16.371799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] PATH_INFO: `/v3/projects/e1194991899d4dd5ac974608165516b8` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:16.374010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1194991899d4dd5ac974608165516b8)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:16.376004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:16.497743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1194991899d4dd5ac974608165516b8', 'request_id': 'req-b007b376-b01b-4020-8a56-00bdc29bd461'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:16.503621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1194991899d4dd5ac974608165516b8', 'request_id': 'req-b007b376-b01b-4020-8a56-00bdc29bd461'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:16.610543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1194991899d4dd5ac974608165516b8', 'request_id': 'req-b007b376-b01b-4020-8a56-00bdc29bd461'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:16.628054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] The token cache is being invalidate because project e1194991899d4dd5ac974608165516b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:33:16.628578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:16.629471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b007b376-b01b-4020-8a56-00bdc29bd461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:16.634749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1163/2294] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:33:16 2022] DELETE /identity/v3/projects/e1194991899d4dd5ac974608165516b8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:33:16.640424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:16.640953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:16.641241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:16.661236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:16.662695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:16.670021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:16.727885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1132/2295] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:16.736367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:16.736670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:16.736959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:16.757534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:16.758701 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:16.767353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1164/2296] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:16.771759 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:16.783580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb50a48-4fc5-40a7-ae0c-f7f008cc6ea9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:16.788334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:16.796754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:16.805530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1133/2297] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:33:16.810775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:16.820843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ceb0-3ae7-4107-b096-27201cc1ed9d None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:16.825750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:16.859821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:16.859821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:16.859821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:16.859821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:16.859821 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:16.986126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:16.993078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1165/2298] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:33:16 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:33:16.998516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:17.009396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861a7c9-aefa-43d0-81dd-22b912e64166 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:17.068988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:17.069685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.069991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.070264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.071012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:17.072878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:17.194205 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1134/2299] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:33:16 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:33:17.197304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:17.209131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380ba5-3df3-4d63-b4a2-fc95d5481c6c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:17.213095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:17.213895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.214096 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.214379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.215117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:17.217525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:17.228430 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1166/2300] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:33:17.244688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:17.255897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cf750-b261-4634-b33e-7d80bf42479a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:17.260649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:17.261404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.261758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.262595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.263678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:17.265567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d19fdf-8cd3-455e-991b-7bfe076121de admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:17.275745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1135/2301] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:33:17.281287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:17.291984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9774-11a8-4181-b459-5b652b8359d2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:17.297733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:17.298618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.298904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.299180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] PATH_INFO: `/v3/projects/91db4a9f7250434c888a32d700180a2c/users/8c3318e2bebb4208a8842c40c60f91c2/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.314989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91db4a9f7250434c888a32d700180a2c, user_id=8c3318e2bebb4208a8842c40c60f91c2, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:17.317898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:17.398686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1167/2302] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:33:17 2022] PUT /identity/v3/projects/91db4a9f7250434c888a32d700180a2c/users/8c3318e2bebb4208a8842c40c60f91c2/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:33:17.409372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.409691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.409980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.438377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] MFA Rules not processed for user `8c3318e2bebb4208a8842c40c60f91c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:17.439637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:17.509916 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1136/2303] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:17.524304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:17.524304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:17.585358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750779e-044b-4f10-9702-a1deb3d36c29 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:17.588942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:17.589616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:17.589902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:17.590222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:17.599285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:17.602212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfe41b-8883-4728-8efd-83a68003a27e service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:17.610217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1168/2304] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:27.506860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:27.605492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:29.864887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:29.957965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6128f-db56-4221-8d51-d48d83cdf4db None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:29.963025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:29.963981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:29.964265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:29.964565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] PATH_INFO: `/v3/users/8c3318e2bebb4208a8842c40c60f91c2` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:29.965926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3318e2bebb4208a8842c40c60f91c2)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:29.968197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:30.274457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3318e2bebb4208a8842c40c60f91c2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:30.292616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3318e2bebb4208a8842c40c60f91c2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:30.421585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3318e2bebb4208a8842c40c60f91c2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:30.422552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3318e2bebb4208a8842c40c60f91c2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:30.423174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3318e2bebb4208a8842c40c60f91c2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:30.437540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1137/2305] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:33:29 2022] DELETE /identity/v3/users/8c3318e2bebb4208a8842c40c60f91c2 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:33:30.442426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:30.442933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:30.443035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:30.464539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:30.466324 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:30.537694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1169/2306] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:30.562563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:30.571145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:30.664739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7321d-9592-4318-92d6-e8614282330a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:30.671345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:30.672240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:30.672536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:30.672956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:30.684386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:30.686920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edd6b4-82ec-47c0-9582-9774e6a5f32c service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:30.699102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1138/2307] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:31.245603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:31.247363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:31.261854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e847b2-1ed0-4c10-ac82-9985aaea1a39 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:31.282439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:31.283576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:31.283852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:31.284290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] PATH_INFO: `/v3/projects/91db4a9f7250434c888a32d700180a2c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:31.286212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91db4a9f7250434c888a32d700180a2c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:31.288197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:31.524029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91db4a9f7250434c888a32d700180a2c', 'request_id': 'req-a830c8df-5658-4842-a643-0474b95618e1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:31.535315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91db4a9f7250434c888a32d700180a2c', 'request_id': 'req-a830c8df-5658-4842-a643-0474b95618e1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:31.703589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91db4a9f7250434c888a32d700180a2c', 'request_id': 'req-a830c8df-5658-4842-a643-0474b95618e1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:31.720432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] The token cache is being invalidate because project 91db4a9f7250434c888a32d700180a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:33:31.721119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:31.721952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:33:31.724175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1170/2308] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:33:31 2022] DELETE /identity/v3/projects/91db4a9f7250434c888a32d700180a2c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:33:33.713510 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:33.713956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:33.714062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:33.733583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] MFA Rules not processed for user `391d41dab9294cd39d3b80d60570034e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:33.734841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:33.752616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:33.792875 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1139/2309] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:39.192434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:39.195122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:39.280145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c8df-5658-4842-a643-0474b95618e1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:39.284427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:39.285284 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:39.285709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:39.286087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:39.288893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:39.331139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:39.363965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:39.365452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:39.376820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1171/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:42.543444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:42.553279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3a115-fa31-48aa-8462-33beec3ca26a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:42.556981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:42.557691 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:42.557974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:42.558251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:42.560118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:42.575564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:42.577003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f54757-2406-479f-8a64-70576bce0462 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:42.588733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1140/2311] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:49.564022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=16 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:33:49.800756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:49.803322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:49.816756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9f524-468b-4b62-a631-7e3e8f1ef841 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:49.822595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:49.823240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:49.823528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:49.823905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:49.825842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:49.837619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:49.839212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:49.849047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1172/2312] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:52.401859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:52.402409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:52.402505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:52.423649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] MFA Rules not processed for user `87d41932bfa94f369103ed29d29940b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:33:52.424443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:52.487394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1141/2313] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:33:52.835996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:52.846155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a32b-ac73-411e-b96a-86a67771a9de service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:52.849888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:52.850530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:52.850814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:52.851094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:52.852872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:52.863418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:52.864889 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:52.874103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1173/2314] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:53.346977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:53.346977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:53.426193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13221c-d26c-44c9-8970-9e706f25909a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:53.431217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:53.431884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:53.432164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:53.432446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:53.434249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:53.447099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:53.448808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:53.458951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1142/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:55.509663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:55.523926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a2770-b85e-4fd9-8fdf-be5d2330e874 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:55.529590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:55.530286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:55.530577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:55.530877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:55.533553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:55.544895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:55.547072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:55.558339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1174/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:57.107855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:57.109301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:57.120834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38531b-4e0d-462a-a558-9eb4e50c98db service placement] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:57.124610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:57.125938 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:57.126362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:57.126522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:57.128255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:57.138136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:57.139726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:57.150452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1143/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:33:58.070567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:33:58.072090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:58.087909 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517d7a6-9394-4273-aeba-615d3e77e1f2 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:33:58.093395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:33:58.095113 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:33:58.095303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:33:58.095585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:33:58.097401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:33:58.158443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:33:58.160065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:33:58.170839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1175/2318] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:33:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:03.373829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:03.389090 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0d75-19c7-41b7-996f-837ac2fd9610 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:03.395337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:03.397362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:03.397362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:03.397362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:03.399496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:03.425076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:03.427430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:03.440063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1144/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:08.165206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=46 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:13.434674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:15.872724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:15.886801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336fe10-2a65-4f5e-a52e-08ac5e624293 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:15.894620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:15.895365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:15.895726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:15.896775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:15.898020 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:15.976624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:15.978327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:15.994607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1176/2320] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:22.491980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:22.501612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9bfcb-88dd-41c6-acb2-b97cc0ac06bc service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:22.505613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:22.506255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:22.506542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:22.506823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:22.508581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:22.571728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:22.573211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:22.583135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1145/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:25.985106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=28 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:26.249792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:26.251306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:26.276685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32cfcb-1455-4900-a05e-b3ab115e6733 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:26.281253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:26.282182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:26.282561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:26.282910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] PATH_INFO: `/v3/users/cf3e0adc32814585b57bf9f884840fc9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:26.284181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3e0adc32814585b57bf9f884840fc9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:26.286186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:26.492583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3e0adc32814585b57bf9f884840fc9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:26.493157 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3e0adc32814585b57bf9f884840fc9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:26.565022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3e0adc32814585b57bf9f884840fc9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:26.565667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3e0adc32814585b57bf9f884840fc9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:26.578325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-63943189-2e0c-4fe7-a8ff-c8705c81dfdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3e0adc32814585b57bf9f884840fc9'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:26.591077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1177/2322] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:26 2022] DELETE /identity/v3/users/cf3e0adc32814585b57bf9f884840fc9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:26.852407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:26.864076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88815c5d-5cc6-454e-a5b9-adebb5ea8e84 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:26.926052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:26.926924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:26.927216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:26.927498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] PATH_INFO: `/v3/projects/6b6ae55c00cb4a43b8e2bc7df2ea9626` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:26.928706 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b6ae55c00cb4a43b8e2bc7df2ea9626)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:26.930879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.039819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b6ae55c00cb4a43b8e2bc7df2ea9626', 'request_id': 'req-a0180c52-02b9-4b0e-941d-72fee77c2f59'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:27.096201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b6ae55c00cb4a43b8e2bc7df2ea9626', 'request_id': 'req-a0180c52-02b9-4b0e-941d-72fee77c2f59'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:27.103052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b6ae55c00cb4a43b8e2bc7df2ea9626', 'request_id': 'req-a0180c52-02b9-4b0e-941d-72fee77c2f59'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:27.115643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] The token cache is being invalidate because project 6b6ae55c00cb4a43b8e2bc7df2ea9626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:27.115939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:27.116593 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a0180c52-02b9-4b0e-941d-72fee77c2f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:27.118411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1146/2323] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:26 2022] DELETE /identity/v3/projects/6b6ae55c00cb4a43b8e2bc7df2ea9626 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:27.123302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.123773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.123893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.141935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:27.143964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:27.203690 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1178/2324] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:27.212197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.212514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.212797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.232547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:27.233991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:27.242680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1147/2325] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:27.246951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.257217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915b26d-b121-4c89-ae4f-fced4695e4e0 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.261870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.262523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.262813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.263097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.263826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:34:27.264139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.265884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.278026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1179/2326] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:34:27.282542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.292480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c12d-b17f-49e0-a6a0-0150ffceba4a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.297121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.297806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.298093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.298376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.299077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.300977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.314814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:27.412559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1148/2327] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:27.416871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.427501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70993f6-6ad3-4822-b59e-1b8e80ec2d76 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.486502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.487165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.487453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.487725 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.488425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.490418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.497053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:27.584580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1180/2328] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/users => generated 302 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:27.587364 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.598454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014eab-2343-4041-95b1-f7c14075ae95 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.603406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.604064 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.604352 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.604633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.605372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.607118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.618124 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1149/2329] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:27.622044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.633963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c50767-ce07-4ddc-9bff-24cd7a1e01ec admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.639620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.639620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.639620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.639895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.640588 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.642820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad9115-1113-4ee6-b56c-f0df40be0fb3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.653283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1181/2330] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:27.657577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.667939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcc2f8-5619-42d6-8b77-73af8defdb43 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.672751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.673686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.673975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.674254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] PATH_INFO: `/v3/projects/c672cea68e6b4921acf9619ecc7ee79e/users/2744ac2785254ef4a3de5b4fb5b1eefe/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.689807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c672cea68e6b4921acf9619ecc7ee79e, user_id=2744ac2785254ef4a3de5b4fb5b1eefe, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.692556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:27.771589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1150/2331] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:27 2022] PUT /identity/v3/projects/c672cea68e6b4921acf9619ecc7ee79e/users/2744ac2785254ef4a3de5b4fb5b1eefe/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:27.781779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.782078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.782364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.808721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] MFA Rules not processed for user `2744ac2785254ef4a3de5b4fb5b1eefe`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:27.810006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:27.878596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1182/2332] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:27.883031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:27.946533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73b861-fe02-42f8-ad3b-b30f97427c8e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:27.951179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:27.951830 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:27.952115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:27.952387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:27.953065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:27.955119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.063287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1151/2333] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:27 2022] POST /identity/v3/projects => generated 403 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:34:28.067935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.078693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d164950-830e-4f77-87a4-9d94e3d0f9c5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.140711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.141418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.141709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.141984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.142797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.144846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.261276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1183/2334] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:28 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:28.265261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.276016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8421dde-8c25-4c4a-bca5-c2504ac0bed4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.280674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.281331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.281674 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.281991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.282715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.285095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.295856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1152/2335] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:28.300985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.311218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94042f41-c5e1-49d7-93a4-767b49080f06 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.315895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.316779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.317069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.317373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] PATH_INFO: `/v3/projects/fa699f7ce592491ba46167706755d2d5/users/46147c5f081546f4972f8bdbc8a844bc/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.334395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa699f7ce592491ba46167706755d2d5, user_id=46147c5f081546f4972f8bdbc8a844bc, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.337248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.439641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1184/2336] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:28 2022] PUT /identity/v3/projects/fa699f7ce592491ba46167706755d2d5/users/46147c5f081546f4972f8bdbc8a844bc/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:28.443872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.453968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6133a-df3a-4190-acfd-ae2dcfc6280d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.512744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.513396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.513793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.514077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.514788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.516387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a86b2-ac47-451d-99e6-6a2e416899b9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.526984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1153/2337] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:28.531116 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.541955 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149b18-3aff-455e-b579-d8608d89c87d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.546616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.547508 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.547792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.548069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] PATH_INFO: `/v3/projects/fa699f7ce592491ba46167706755d2d5/users/46147c5f081546f4972f8bdbc8a844bc/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.550063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa699f7ce592491ba46167706755d2d5, user_id=46147c5f081546f4972f8bdbc8a844bc, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.552733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.562815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1185/2338] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:28 2022] PUT /identity/v3/projects/fa699f7ce592491ba46167706755d2d5/users/46147c5f081546f4972f8bdbc8a844bc/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:28.574488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.581934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.581934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.602835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] MFA Rules not processed for user `46147c5f081546f4972f8bdbc8a844bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:28.604143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:28.622837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:28.672039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1154/2339] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:28.683403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:28.685124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:28.707697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:28.743500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafae9-76a9-42b4-a708-535e8357045b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:28.747149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:28.747791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:28.748331 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:28.748607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:28.758306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:28.759780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fe8d1-9ee0-4d95-9e04-fd9b4ad56f2f service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:28.769168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1186/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:28.939738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:29.008288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c202f-f55b-47f1-b231-80f4f7f65dbd None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:29.012953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:29.021568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:29.021568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:29.021568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] PATH_INFO: `/v3/users/2744ac2785254ef4a3de5b4fb5b1eefe` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:29.021568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2744ac2785254ef4a3de5b4fb5b1eefe)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:29.021568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:29.277615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2744ac2785254ef4a3de5b4fb5b1eefe'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:29.294591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2744ac2785254ef4a3de5b4fb5b1eefe'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:29.375686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2744ac2785254ef4a3de5b4fb5b1eefe'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:29.376312 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2744ac2785254ef4a3de5b4fb5b1eefe'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:29.377774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2744ac2785254ef4a3de5b4fb5b1eefe'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:29.391224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1155/2341] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:28 2022] DELETE /identity/v3/users/2744ac2785254ef4a3de5b4fb5b1eefe => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:29.398263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:29.398263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:29.398263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:29.416173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:29.417425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:29.477093 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1187/2342] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:29.486817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:29.488378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:29.530767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:29.548405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba984e-6f99-4f24-a74c-db00410ccc13 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:29.553869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:29.554535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:29.554850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:29.555131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:29.565279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:29.567555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:29.577590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1156/2343] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:30.000456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:30.001917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:30.019376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a0ee2-dab9-444c-a80a-b035a7f563c7 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:30.021995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:30.022892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:30.023181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:30.023461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] PATH_INFO: `/v3/projects/c672cea68e6b4921acf9619ecc7ee79e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:30.024686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c672cea68e6b4921acf9619ecc7ee79e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:30.026657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:30.155459 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c672cea68e6b4921acf9619ecc7ee79e', 'request_id': 'req-27a3502d-b8fc-4847-86ea-4200788e9d16'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.161658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c672cea68e6b4921acf9619ecc7ee79e', 'request_id': 'req-27a3502d-b8fc-4847-86ea-4200788e9d16'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.250141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c672cea68e6b4921acf9619ecc7ee79e', 'request_id': 'req-27a3502d-b8fc-4847-86ea-4200788e9d16'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.262610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] The token cache is being invalidate because project c672cea68e6b4921acf9619ecc7ee79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:30.262906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.263558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.265176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1188/2344] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:29 2022] DELETE /identity/v3/projects/c672cea68e6b4921acf9619ecc7ee79e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:30.269402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:30.279381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e058e7-d212-4584-ab42-04d3439eb9c0 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:30.338729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:30.339528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:30.339813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:30.340094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] PATH_INFO: `/v3/users/46147c5f081546f4972f8bdbc8a844bc` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:30.341731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=46147c5f081546f4972f8bdbc8a844bc)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:30.343822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:30.557970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46147c5f081546f4972f8bdbc8a844bc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.571505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46147c5f081546f4972f8bdbc8a844bc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.695846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46147c5f081546f4972f8bdbc8a844bc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.696601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46147c5f081546f4972f8bdbc8a844bc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.697217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46147c5f081546f4972f8bdbc8a844bc'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:30.711663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1157/2345] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:30 2022] DELETE /identity/v3/users/46147c5f081546f4972f8bdbc8a844bc => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:30.873635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:30.873635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:30.973703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:31.052527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3502d-b8fc-4847-86ea-4200788e9d16 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:31.156939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:31.158732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.159755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.160139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.162428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:31.190449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:31.193884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f91609-fc5e-4fa1-80e7-405e41ce06fc service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:31.207087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1189/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:31.320614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:31.325016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:31.408507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:31.413752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c91e-dc6b-493f-9d27-00fd02d765ad admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:31.418694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:31.419887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.420224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.420936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] PATH_INFO: `/v3/projects/fa699f7ce592491ba46167706755d2d5` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.422258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa699f7ce592491ba46167706755d2d5)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:31.424194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:31.525913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa699f7ce592491ba46167706755d2d5', 'request_id': 'req-df3abde2-5c72-42d8-9158-b4969c82f071'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:31.594004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa699f7ce592491ba46167706755d2d5', 'request_id': 'req-df3abde2-5c72-42d8-9158-b4969c82f071'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:31.600091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa699f7ce592491ba46167706755d2d5', 'request_id': 'req-df3abde2-5c72-42d8-9158-b4969c82f071'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:31.611450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] The token cache is being invalidate because project fa699f7ce592491ba46167706755d2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:31.611774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:31.613195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-df3abde2-5c72-42d8-9158-b4969c82f071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:31.614687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1158/2347] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:31 2022] DELETE /identity/v3/projects/fa699f7ce592491ba46167706755d2d5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:31.620008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccb4f-1856-4c89-8862-860881754633 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.620303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccb4f-1856-4c89-8862-860881754633 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.620589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ccb4f-1856-4c89-8862-860881754633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.654246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-066ccb4f-1856-4c89-8862-860881754633 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:31.654246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-066ccb4f-1856-4c89-8862-860881754633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:31.709520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1190/2348] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:31.717328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.718417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.718791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.740448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:31.741818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:31.753701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1159/2349] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:31.755850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccb4f-1856-4c89-8862-860881754633 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:31.766193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ccb4f-1856-4c89-8862-860881754633 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:31.771883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:31.773680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.773680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.773680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.773979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:34:31.774360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:31.776012 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:31.788779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1191/2350] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:34:31.793198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:31.803629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa124f1-1933-414d-b20d-021bcb145ebd None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:31.809028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:31.809757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.810054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.810328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.811029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:31.812951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:31.914739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1160/2351] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:31 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:34:31.918849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:31.930445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef192d-4137-4e24-ae70-74cf706849f4 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:31.990042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:31.995370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:31.996038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:31.996323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:31.996598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:31.998367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:31.999711 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:32.083401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1192/2352] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:31 2022] POST /identity/v3/users => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:32.087919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:32.100749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26899470-cbba-4fec-8da6-792279a0c07f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:32.112074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:32.113330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:32.113645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:32.120489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:32.120489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:32.120489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:32.128027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1161/2353] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:32.132083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:32.145520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef5bb3-3429-40cb-98fc-c3cb094d8d1f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:32.150972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:32.154919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:32.154919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:32.154919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:32.154919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:32.154919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10d89e-d31a-4e20-9e9a-83e96e387fdc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:32.169703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1193/2354] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:32.171088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:32.183852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe267e-d8ef-464c-99b3-2ef1c5ab2f39 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:32.189239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:32.190363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:32.190647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:32.190933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] PATH_INFO: `/v3/projects/fb4046cc410546efb8bf73a9021ea998/users/ce077bf7fd6a4a7896104663804afb0e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:32.208648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4046cc410546efb8bf73a9021ea998, user_id=ce077bf7fd6a4a7896104663804afb0e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:32.212223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:32.319776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1162/2355] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:32 2022] PUT /identity/v3/projects/fb4046cc410546efb8bf73a9021ea998/users/ce077bf7fd6a4a7896104663804afb0e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:32.325036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a40a-ebab-4743-9251-d6e8a526aa5b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:32.325334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a40a-ebab-4743-9251-d6e8a526aa5b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:32.325639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a40a-ebab-4743-9251-d6e8a526aa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:32.345946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5615a40a-ebab-4743-9251-d6e8a526aa5b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:32.347992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5615a40a-ebab-4743-9251-d6e8a526aa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:32.419345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1194/2356] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:32.429902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:32.431687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:32.501455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac903cb-b89e-4abc-9ed7-ed880fa955d7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:32.507420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:32.508184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:32.508564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:32.508913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:32.520465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:32.522725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:32.555300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1163/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:34:33.969529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:33.970355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:33.970796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.008314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] MFA Rules not processed for user `ce077bf7fd6a4a7896104663804afb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:34.010011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:34.092126 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1195/2358] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:34.097153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.107048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17de2-910e-4cf5-a44f-f6d7cf1a9415 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.111620 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.112432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.112681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.113015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.113742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.115869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.256880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1164/2359] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:34 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:34.261047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.276329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258daaa-ff31-44a2-9aac-e2cfd0a10d40 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.340231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.341064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.341375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.341663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.342356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.344285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.354596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:34.451991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1196/2360] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:34 2022] POST /identity/v3/users => generated 308 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:34:34.456127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.462820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:34.466564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7a885-27b6-4e56-823b-570667182b5b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.471093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.472388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.472675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.472954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.473723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.475328 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.485849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1165/2361] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:34.490733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.501140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162617-4df4-4efb-9c7c-cb18c3d1d285 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.506364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.507287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.507525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.507802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] PATH_INFO: `/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3/users/575687635c7e4394a0e3f752b2cef31b/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.523559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c4abae0a6a437e98f4cfd4bfeec3f3, user_id=575687635c7e4394a0e3f752b2cef31b, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.526283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.616926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1197/2362] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:34 2022] PUT /identity/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3/users/575687635c7e4394a0e3f752b2cef31b/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:34.621036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.631416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f718515-fecf-4017-ae4b-fb93847ac4ec admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.697286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.698001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.698295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.698574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.699302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.701051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f672a6-670a-4192-a815-be0dad952525 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.712899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1166/2363] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:34.717140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:34.728839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648a9de-19c3-46d2-b251-6e2a1d2ac963 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:34.733732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:34.735235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:34.735594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:34.735980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] PATH_INFO: `/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3/users/575687635c7e4394a0e3f752b2cef31b/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:34.738866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c4abae0a6a437e98f4cfd4bfeec3f3, user_id=575687635c7e4394a0e3f752b2cef31b, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:34.741999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:34.755231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1198/2364] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:34 2022] PUT /identity/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3/users/575687635c7e4394a0e3f752b2cef31b/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:36.639669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:36.640601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:36.641410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:36.686805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] MFA Rules not processed for user `575687635c7e4394a0e3f752b2cef31b`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:36.689649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:36.779910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1167/2365] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:38.782697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:38.847790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77c40-9674-48ff-bbb3-30d951db1cf3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:38.852635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:38.853530 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:38.853816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:38.859641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] PATH_INFO: `/v3/users/ce077bf7fd6a4a7896104663804afb0e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:38.859641 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce077bf7fd6a4a7896104663804afb0e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:38.860123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:39.104509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce077bf7fd6a4a7896104663804afb0e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.105651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce077bf7fd6a4a7896104663804afb0e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.204699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce077bf7fd6a4a7896104663804afb0e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.205312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce077bf7fd6a4a7896104663804afb0e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.219083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce077bf7fd6a4a7896104663804afb0e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.232988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1199/2366] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:38 2022] DELETE /identity/v3/users/ce077bf7fd6a4a7896104663804afb0e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:39.598903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:39.626095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb904c8-b03b-4640-bae9-c7bc36e55f84 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:39.693741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:39.694621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:39.694908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:39.695243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] PATH_INFO: `/v3/projects/fb4046cc410546efb8bf73a9021ea998` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:39.696457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4046cc410546efb8bf73a9021ea998)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:39.698506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:39.716197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:39.828592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4046cc410546efb8bf73a9021ea998', 'request_id': 'req-a2d79f62-5816-4bfb-a905-51031102bf35'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.935189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4046cc410546efb8bf73a9021ea998', 'request_id': 'req-a2d79f62-5816-4bfb-a905-51031102bf35'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.942817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4046cc410546efb8bf73a9021ea998', 'request_id': 'req-a2d79f62-5816-4bfb-a905-51031102bf35'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.957483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] The token cache is being invalidate because project fb4046cc410546efb8bf73a9021ea998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:39.957746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.958396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:39.959918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1168/2367] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:39 2022] DELETE /identity/v3/projects/fb4046cc410546efb8bf73a9021ea998 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:39.964050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:39.965595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:39.978992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fbad2-ed60-4104-b5fa-9c9f2c62d7de admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:40.014353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:40.037844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:40.038785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:40.039073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:40.039353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] PATH_INFO: `/v3/users/575687635c7e4394a0e3f752b2cef31b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:40.040564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=575687635c7e4394a0e3f752b2cef31b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:40.042669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:40.244077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575687635c7e4394a0e3f752b2cef31b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:40.245070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575687635c7e4394a0e3f752b2cef31b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:40.330465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575687635c7e4394a0e3f752b2cef31b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:40.331467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575687635c7e4394a0e3f752b2cef31b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:40.344464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-218f6513-7d01-4047-8cc8-e9fcabbd1487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575687635c7e4394a0e3f752b2cef31b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:40.358301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1200/2368] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:39 2022] DELETE /identity/v3/users/575687635c7e4394a0e3f752b2cef31b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:40.636954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:40.638438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:40.652303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d79f62-5816-4bfb-a905-51031102bf35 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:40.726688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:40.728043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:40.728043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:40.728263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] PATH_INFO: `/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:40.729294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c4abae0a6a437e98f4cfd4bfeec3f3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:40.731416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:40.922479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c4abae0a6a437e98f4cfd4bfeec3f3', 'request_id': 'req-b71db0d8-f11a-4bce-9f0f-740198278026'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:41.040276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c4abae0a6a437e98f4cfd4bfeec3f3', 'request_id': 'req-b71db0d8-f11a-4bce-9f0f-740198278026'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:41.052399 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c4abae0a6a437e98f4cfd4bfeec3f3', 'request_id': 'req-b71db0d8-f11a-4bce-9f0f-740198278026'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:41.059700 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] The token cache is being invalidate because project e3c4abae0a6a437e98f4cfd4bfeec3f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:41.059991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:41.060645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b71db0d8-f11a-4bce-9f0f-740198278026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:41.062239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1169/2369] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:40 2022] DELETE /identity/v3/projects/e3c4abae0a6a437e98f4cfd4bfeec3f3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:41.067444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.067920 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.068014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.089553 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:41.090907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:41.182703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1201/2370] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:41.193728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.193728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.193728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.215991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:41.217258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:41.245461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1170/2371] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:34:41.246721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.261956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045de6e7-2f4f-45a7-970e-b4b76b24555a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.267555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.268443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.268837 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.269231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.270107 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:34:41.270540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.272706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.290304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1202/2372] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:34:41.294540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.305639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c47c-bc5e-4858-a956-2f948e8bcb58 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.311888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.313418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.313418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.313636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.319122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.319122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.445711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1171/2373] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:41.446967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.460279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409947a-4563-45e6-89b1-39fe5e2830d4 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.516097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:41.532291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.533194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.533521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.533805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.535101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.537138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.607325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1203/2374] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:41.611410 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.624825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d1415-2d62-4196-ace4-1593d117296e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.633150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.634006 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.635018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.635378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.637182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.638868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.640424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:41.655544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1172/2375] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:41.657060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.669190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c425e-3cc9-42f5-b3b1-e031bd831019 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.674507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.675279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.675734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.676043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.676764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.678477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e11be5-82ca-4c9a-9c82-28c057f9f72b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.689983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1204/2376] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:41.694143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:41.704273 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef674a-b0d9-47d4-8fa8-8c6c337d9cd7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:41.710371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:41.711286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.711574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.711862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] PATH_INFO: `/v3/projects/423e93e3d2644ec68ed48847a1835159/users/7436d2012e01445d9aaebc5a4dc7dfc2/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.728477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423e93e3d2644ec68ed48847a1835159, user_id=7436d2012e01445d9aaebc5a4dc7dfc2, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:41.731239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:41.825049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1173/2377] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:41 2022] PUT /identity/v3/projects/423e93e3d2644ec68ed48847a1835159/users/7436d2012e01445d9aaebc5a4dc7dfc2/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:41.836343 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:41.836637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:41.836918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:41.864432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] MFA Rules not processed for user `7436d2012e01445d9aaebc5a4dc7dfc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:41.865718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:41.965591 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1205/2378] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:41.970367 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.056125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc866a5-fc5e-435b-aa16-2cdb945626b4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.061035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.061814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.062173 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.062530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.063320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.065939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3c343-587b-4b05-81fa-eee74509f23b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.168544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1174/2379] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:34:41 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:34:42.173154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.184539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aca60-0586-4d71-9cc2-565733afdb32 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.256110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.256938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.257239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.257747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.258638 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.261578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.328110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.328409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.328695 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.347590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] MFA Rules not processed for user `56799d191ffe4193918fa0221c27c0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:42.348816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:42.377007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1206/2380] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:34:42 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:34:42.382361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.393037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4dacb-0447-45e9-a14f-60b11068720f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.398931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.399667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.399975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.400260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.400992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.402654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.408229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1175/2381] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:34:42.419654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1207/2382] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:34:42.423819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.435411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153786f3-070a-4613-a6b2-a77a2d75d051 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.441418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.441970 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.442258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.442595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] PATH_INFO: `/v3/projects/dbf8a388d9d54966b46266fad7cd58e1/users/d8f04ddcd03e4471b11f9d860927514f/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.460258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf8a388d9d54966b46266fad7cd58e1, user_id=d8f04ddcd03e4471b11f9d860927514f, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.463063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.566095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1176/2383] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:42 2022] PUT /identity/v3/projects/dbf8a388d9d54966b46266fad7cd58e1/users/d8f04ddcd03e4471b11f9d860927514f/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:42.570101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.581576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a389be4-dad9-4fc6-94c3-c7563832f53d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.614557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:42.645091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.645913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.646201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.646475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.647197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.649569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc650-7c55-4625-bd1f-9709bf870318 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.661109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1208/2384] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:34:42.665168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.677549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a60514-9c29-4340-ab59-d3d4f74810b7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.682377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.683468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.683468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.683766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] PATH_INFO: `/v3/projects/dbf8a388d9d54966b46266fad7cd58e1/users/d8f04ddcd03e4471b11f9d860927514f/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.685851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf8a388d9d54966b46266fad7cd58e1, user_id=d8f04ddcd03e4471b11f9d860927514f, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.688520 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.700228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1177/2385] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:34:42 2022] PUT /identity/v3/projects/dbf8a388d9d54966b46266fad7cd58e1/users/d8f04ddcd03e4471b11f9d860927514f/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:42.712182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.712477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.712790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.743133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] MFA Rules not processed for user `d8f04ddcd03e4471b11f9d860927514f`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:42.744495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:42.751092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.752840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:42.782853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:42.814729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077308f-c114-42de-bf67-988e841093a7 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.819125 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.819793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.820081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.820362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.822350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:42.822644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1209/2386] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:42.832980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.834805 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.834922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:42.836226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:42.843886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1178/2387] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:34:42.897761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1967f-7796-4cba-935b-dd811c1bab6e None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:42.901599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:42.902286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:42.902576 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:42.902856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:42.913228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:42.914854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de01fd-15c7-477e-8f5a-8e52844feec3 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:42.926053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1210/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:34:43.147814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:43.158177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85518cb0-af6b-4dbb-8518-5eae9b19cb0a service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:43.162786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:43.163439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:43.163838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:43.164120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:43.249905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:43.251490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:43.265690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1179/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:43.358812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dde8b-c236-4fc0-bdf9-2cdcdf3fed33 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:43.359263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dde8b-c236-4fc0-bdf9-2cdcdf3fed33 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:43.359386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dde8b-c236-4fc0-bdf9-2cdcdf3fed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:43.379387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-664dde8b-c236-4fc0-bdf9-2cdcdf3fed33 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:43.380659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-664dde8b-c236-4fc0-bdf9-2cdcdf3fed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:43.442902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:43.450602 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1211/2390] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:43.649660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:43.651141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:43.731658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb4dd-9b9d-45db-8aa6-0ed04a4a24f5 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:43.737986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:43.739016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:43.739016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:43.739279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:43.750482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:43.752113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:43.761683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1180/2391] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:34:53.758609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=75 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:55.287159 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc3c70-b988-4153-8215-15586aace245 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:55.287651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc3c70-b988-4153-8215-15586aace245 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:55.287748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc3c70-b988-4153-8215-15586aace245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:55.306461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-64fc3c70-b988-4153-8215-15586aace245 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:55.307691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-64fc3c70-b988-4153-8215-15586aace245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:55.315677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1212/2392] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:55.330492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:55.344415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ce2fc-cccd-4628-9c9c-4fb2842d3511 service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:55.352881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:55.358525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:55.358525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:55.358525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:55.364041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:55.365789 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:55.374628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1181/2393] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:34:55.399978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc3c70-b988-4153-8215-15586aace245 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:55.401423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-64fc3c70-b988-4153-8215-15586aace245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:55.501893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc3c70-b988-4153-8215-15586aace245 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:55.511248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:55.512505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:55.512800 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:55.513518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] PATH_INFO: `/v3/users/650592005e10426b95a76ebd97b07455` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:55.528652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] RBAC: Authorizing `identity:delete_user(user_id=650592005e10426b95a76ebd97b07455)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:55.547905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:55.790173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '650592005e10426b95a76ebd97b07455'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:55.791057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '650592005e10426b95a76ebd97b07455'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:55.839652 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:55.841460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:55.874312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '650592005e10426b95a76ebd97b07455'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:55.874959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '650592005e10426b95a76ebd97b07455'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:55.937948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '650592005e10426b95a76ebd97b07455'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:55.972465 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1213/2394] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:55 2022] DELETE /identity/v3/users/650592005e10426b95a76ebd97b07455 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:55.985747 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a82a-1d6c-4298-a60c-c1e48a83013c service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:56.140506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:56.141422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:56.141832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:56.142675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:56.147919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:56.166753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:56.168993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:56.196999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1182/2395] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:56.449106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:56.451446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:56.629607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325b2e-27b2-41de-9c43-d8d13217969a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:56.643087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:56.644332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:56.644803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:56.645503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] PATH_INFO: `/v3/projects/bf8f192a31204d97b3b2e8895856a459` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:56.649774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8f192a31204d97b3b2e8895856a459)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:56.652069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:56.678723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:56.855240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8f192a31204d97b3b2e8895856a459', 'request_id': 'req-68c0cd51-603f-402e-b031-a8114e5089f3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:56.861234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8f192a31204d97b3b2e8895856a459', 'request_id': 'req-68c0cd51-603f-402e-b031-a8114e5089f3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:56.953799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8f192a31204d97b3b2e8895856a459', 'request_id': 'req-68c0cd51-603f-402e-b031-a8114e5089f3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:56.971097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] The token cache is being invalidate because project bf8f192a31204d97b3b2e8895856a459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:56.971516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:56.972501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:56.974235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1214/2396] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:56 2022] DELETE /identity/v3/projects/bf8f192a31204d97b3b2e8895856a459 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:56.978629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:57.001087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b1c08-ed3a-4f90-a65e-bf31571ea886 service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:57.112588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:57.114422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:57.114874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:57.115224 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] PATH_INFO: `/v3/users/7c749c3b281142fcb674df9193d45f07` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:57.116574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c749c3b281142fcb674df9193d45f07)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:57.118563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:57.135643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:57.354122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c749c3b281142fcb674df9193d45f07'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:57.369395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c749c3b281142fcb674df9193d45f07'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:57.445408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c749c3b281142fcb674df9193d45f07'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:57.446126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c749c3b281142fcb674df9193d45f07'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:57.446744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c749c3b281142fcb674df9193d45f07'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:57.467568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1183/2397] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:56 2022] DELETE /identity/v3/users/7c749c3b281142fcb674df9193d45f07 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:57.813794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:57.818523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:57.856048 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0cd51-603f-402e-b031-a8114e5089f3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:58.012364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:58.013360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:58.013764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:58.014881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] PATH_INFO: `/v3/projects/f51f25652579464588c765d769165de8` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:58.021498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51f25652579464588c765d769165de8)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:58.025333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:58.027577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:58.029817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:58.043648 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b48c0b-122e-420d-be30-91c4f326e975 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:58.052533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:58.054171 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:58.054650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:58.055554 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] PATH_INFO: `/v3/users/7436d2012e01445d9aaebc5a4dc7dfc2` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:58.057411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7436d2012e01445d9aaebc5a4dc7dfc2)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:58.060606 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:58.166838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51f25652579464588c765d769165de8', 'request_id': 'req-8fce41b6-0899-4f44-9901-da06ed41929e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.174208 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51f25652579464588c765d769165de8', 'request_id': 'req-8fce41b6-0899-4f44-9901-da06ed41929e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.296463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51f25652579464588c765d769165de8', 'request_id': 'req-8fce41b6-0899-4f44-9901-da06ed41929e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.310001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7436d2012e01445d9aaebc5a4dc7dfc2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.316815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] The token cache is being invalidate because project f51f25652579464588c765d769165de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:58.317112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.318339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8fce41b6-0899-4f44-9901-da06ed41929e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.333428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1215/2398] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:57 2022] DELETE /identity/v3/projects/f51f25652579464588c765d769165de8 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:58.341130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7436d2012e01445d9aaebc5a4dc7dfc2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.428631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7436d2012e01445d9aaebc5a4dc7dfc2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.433968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7436d2012e01445d9aaebc5a4dc7dfc2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.433968 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7436d2012e01445d9aaebc5a4dc7dfc2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:58.466576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1184/2399] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:58 2022] DELETE /identity/v3/users/7436d2012e01445d9aaebc5a4dc7dfc2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:34:58.470835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:58.472694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:58.473322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:58.528715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:34:58.530724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:58.586636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:58.641734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1216/2400] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:34:58.656532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:58.658315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:58.755645 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e52bcc-811a-4f6f-a548-9000a62118ba admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:58.760160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:58.760947 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:58.761955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:58.761955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:58.777102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:58.778739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:58.795509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1185/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:34:59.511639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:59.513238 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:34:59.526907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3866a6-d087-44ed-ad15-377eedde8c44 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:59.532235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:59.533114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:59.533485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:59.533768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] PATH_INFO: `/v3/projects/423e93e3d2644ec68ed48847a1835159` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:59.544241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=423e93e3d2644ec68ed48847a1835159)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:59.544241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:34:59.689289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423e93e3d2644ec68ed48847a1835159', 'request_id': 'req-71d61582-12e0-42a8-b522-1c725bb3bcbe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:59.697869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423e93e3d2644ec68ed48847a1835159', 'request_id': 'req-71d61582-12e0-42a8-b522-1c725bb3bcbe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:59.770974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423e93e3d2644ec68ed48847a1835159', 'request_id': 'req-71d61582-12e0-42a8-b522-1c725bb3bcbe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:59.784820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] The token cache is being invalidate because project 423e93e3d2644ec68ed48847a1835159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:34:59.785128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:59.786937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:34:59.793057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1217/2402] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:34:59 2022] DELETE /identity/v3/projects/423e93e3d2644ec68ed48847a1835159 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:34:59.794476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:34:59.802238 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:34:59.805992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa87e-e70c-41dc-9503-790e05413e3d service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:34:59.881380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:34:59.882726 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:34:59.883014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:34:59.883303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] PATH_INFO: `/v3/users/d8f04ddcd03e4471b11f9d860927514f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:34:59.884514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f04ddcd03e4471b11f9d860927514f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:34:59.886512 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:00.108879 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f04ddcd03e4471b11f9d860927514f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:00.124901 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f04ddcd03e4471b11f9d860927514f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:00.223075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f04ddcd03e4471b11f9d860927514f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:00.223796 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f04ddcd03e4471b11f9d860927514f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:00.224604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-40fc79e0-10f2-48c9-b2e3-e6e412b50e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f04ddcd03e4471b11f9d860927514f'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:00.240260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1186/2403] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:34:59 2022] DELETE /identity/v3/users/d8f04ddcd03e4471b11f9d860927514f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:00.847010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:00.848523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:00.864337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d61582-12e0-42a8-b522-1c725bb3bcbe admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:00.962480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:00.963369 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:00.963748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:00.964031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] PATH_INFO: `/v3/projects/dbf8a388d9d54966b46266fad7cd58e1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:00.965245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf8a388d9d54966b46266fad7cd58e1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:00.967467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:01.141799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf8a388d9d54966b46266fad7cd58e1', 'request_id': 'req-9f0dbca1-0075-49d5-a9aa-997bbd854a49'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:01.151989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf8a388d9d54966b46266fad7cd58e1', 'request_id': 'req-9f0dbca1-0075-49d5-a9aa-997bbd854a49'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:01.155364 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:01.222980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf8a388d9d54966b46266fad7cd58e1', 'request_id': 'req-9f0dbca1-0075-49d5-a9aa-997bbd854a49'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:01.236336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] The token cache is being invalidate because project dbf8a388d9d54966b46266fad7cd58e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:01.236944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:01.237851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9f0dbca1-0075-49d5-a9aa-997bbd854a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:01.244183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1218/2404] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:00 2022] DELETE /identity/v3/projects/dbf8a388d9d54966b46266fad7cd58e1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:01.245709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.246196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.246303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.270076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:01.271299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:01.354583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1187/2405] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:01.357654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.357654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.357654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.380605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:01.381882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:01.390998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1219/2406] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:01.396220 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:01.416185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb32532-6515-471b-9b65-603f9559cd39 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:01.426251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:01.427155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.427462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.427857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.429135 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:01.429756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:01.431872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:01.451362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1188/2407] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:35:01.452499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:01.463785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6fe02-1c07-4267-be79-be774ba06267 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:01.472888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:01.473686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.473974 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.474260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.475189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:01.477143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:01.614478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1220/2408] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:01 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:01.618573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:01.643907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03431eaa-a43d-4e82-82dc-3bd38893f960 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:01.662559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:01.800257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:01.801068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.802088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.802088 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.802529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:01.804670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:01.906825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1189/2409] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:01 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:01.908794 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:01.922984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff01e34-410b-4064-8f27-535660ed57fd admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:01.934655 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:01.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:01.942792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:01.963467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1221/2410] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:01.964642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:01.978581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce285f-ac4f-43b2-8b9c-74a86cddb79b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:01.993909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:01.994804 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:01.995185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:01.995563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:01.996543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:01.998765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81741708-62a0-45b9-bb00-83257704d859 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:02.012874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1190/2411] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:35:02.015967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:02.029270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311699d-03a1-42c4-bdeb-3d96c5cfa7e1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:02.034439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:02.035608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:02.035994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:02.036992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] PATH_INFO: `/v3/projects/35bbeb023d50476aaed88d318a235c95/users/64b2170cff0540a99ed9c865b03c80ec/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:02.067117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bbeb023d50476aaed88d318a235c95, user_id=64b2170cff0540a99ed9c865b03c80ec, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:02.070391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:02.170782 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1222/2412] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:02 2022] PUT /identity/v3/projects/35bbeb023d50476aaed88d318a235c95/users/64b2170cff0540a99ed9c865b03c80ec/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:02.172939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:02.173768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:02.174456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:02.218955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] MFA Rules not processed for user `64b2170cff0540a99ed9c865b03c80ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:02.220687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:02.321380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1191/2413] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:02.326691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:02.405151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac4d9-b3e1-4fac-819f-78edf6e4fd16 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:02.410175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:02.410942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:02.411227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:02.411527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:02.412214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:02.414211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:02.984287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1223/2414] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:02 2022] POST /identity/v3/projects => generated 393 bytes in 658 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:02.989037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:02.999951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd4959-f39b-4040-944e-77a1731e5e72 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:03.062792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:03.063480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.063797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.064104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.064802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:03.067266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:03.199154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1192/2415] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:02 2022] POST /identity/v3/users => generated 303 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:03.200409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:03.246663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8818f-47bf-4c34-8d28-a2c68a74f15a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:03.277597 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:03.279155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.285763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.290095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.295181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:03.301443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:03.314834 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1224/2416] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:03.319712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:03.332032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd100d-e8de-4904-ada5-47845fd3d367 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:03.344487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:03.347984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.347984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.347984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] PATH_INFO: `/v3/projects/769cca51265f4a1b9d6047809a06a487/users/035bd0a4d9754300abb3c594a8593635/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.355890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:03.384245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769cca51265f4a1b9d6047809a06a487, user_id=035bd0a4d9754300abb3c594a8593635, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:03.392503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:03.514194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1193/2417] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:03 2022] PUT /identity/v3/projects/769cca51265f4a1b9d6047809a06a487/users/035bd0a4d9754300abb3c594a8593635/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:03.515480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:03.529254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca1675-fb1c-47d3-954d-4906a8d75f99 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:03.616440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:03.631659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:03.632692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.633311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.633732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.635779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:03.638908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba293070-dcfb-4f19-8121-c8e11ceae693 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:03.655148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1225/2418] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:03.660218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:03.683453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64089c27-3bfd-48bd-8c2b-635dbbd805b5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:03.683453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:03.691104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.691569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.691860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] PATH_INFO: `/v3/projects/769cca51265f4a1b9d6047809a06a487/users/035bd0a4d9754300abb3c594a8593635/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.694303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=769cca51265f4a1b9d6047809a06a487, user_id=035bd0a4d9754300abb3c594a8593635, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:03.698924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:03.729411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1194/2419] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:03 2022] PUT /identity/v3/projects/769cca51265f4a1b9d6047809a06a487/users/035bd0a4d9754300abb3c594a8593635/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:03.733502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:03.733761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:03.733985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:03.792234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] MFA Rules not processed for user `035bd0a4d9754300abb3c594a8593635`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:03.794114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:03.957585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1226/2420] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:03.969579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:03.971555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:04.115897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35c37a-68be-4071-8452-87f4f89fc0bf admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:04.124338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:04.126189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:04.127814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:04.128951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:04.148140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:04.151817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:04.164678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1195/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:04.243942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:04.277883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417f4b7-1f45-44bb-b4db-aa8adf2ee3e2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:04.288189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:04.289949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:04.291772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:04.293294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:04.384226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:04.386189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebf08f-1743-47ea-a5ef-9b50e3fa1142 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:04.397130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1227/2422] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:10.352839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:10.421375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00cdfb-bcb7-48fe-b89c-32277e1b8544 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:10.426590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:10.427478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:10.427802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:10.428093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] PATH_INFO: `/v3/users/64b2170cff0540a99ed9c865b03c80ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:10.429372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b2170cff0540a99ed9c865b03c80ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:10.431348 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:10.665310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b2170cff0540a99ed9c865b03c80ec'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:10.678878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b2170cff0540a99ed9c865b03c80ec'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:10.776468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b2170cff0540a99ed9c865b03c80ec'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:10.777104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b2170cff0540a99ed9c865b03c80ec'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:10.777754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b2170cff0540a99ed9c865b03c80ec'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:10.783275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:10.792106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1196/2423] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:10 2022] DELETE /identity/v3/users/64b2170cff0540a99ed9c865b03c80ec => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:35:10.796721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:10.797217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:10.797329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:10.817213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:10.818609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:10.885199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1228/2424] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:10.895015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:10.896341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:10.955908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959259-f94f-406d-a33b-6b4b59466c5d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:10.960784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:10.961475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:10.961764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:10.962043 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:10.971176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:10.972963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:10.982393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1197/2425] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:35:11.389744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:11.391323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:11.405175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac35790-bba9-4000-8cd3-90c004730380 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:11.411041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:11.413057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:11.413057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:11.413057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] PATH_INFO: `/v3/projects/35bbeb023d50476aaed88d318a235c95` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:11.413996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35bbeb023d50476aaed88d318a235c95)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:11.416258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:11.418883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:11.554084 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35bbeb023d50476aaed88d318a235c95', 'request_id': 'req-07048e5f-666c-4655-9c10-b4f19e781685'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:11.560382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35bbeb023d50476aaed88d318a235c95', 'request_id': 'req-07048e5f-666c-4655-9c10-b4f19e781685'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:11.633890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35bbeb023d50476aaed88d318a235c95', 'request_id': 'req-07048e5f-666c-4655-9c10-b4f19e781685'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:11.646443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] The token cache is being invalidate because project 35bbeb023d50476aaed88d318a235c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:11.646735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:11.647376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:11.648896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1229/2426] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:11 2022] DELETE /identity/v3/projects/35bbeb023d50476aaed88d318a235c95 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:11.652791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:11.663599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf1884-5384-48fb-9edd-8be62962c385 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:11.719829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:11.720602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:11.720880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:11.721150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] PATH_INFO: `/v3/users/035bd0a4d9754300abb3c594a8593635` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:11.723479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] RBAC: Authorizing `identity:delete_user(user_id=035bd0a4d9754300abb3c594a8593635)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:11.725460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:12.014643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035bd0a4d9754300abb3c594a8593635'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.026955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035bd0a4d9754300abb3c594a8593635'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.170597 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035bd0a4d9754300abb3c594a8593635'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.171227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035bd0a4d9754300abb3c594a8593635'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.171891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4ed320ee-6b88-49bb-bfe4-3eef3bf7993a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035bd0a4d9754300abb3c594a8593635'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.185756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1198/2427] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:11 2022] DELETE /identity/v3/users/035bd0a4d9754300abb3c594a8593635 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:12.613253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:12.615083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:12.628827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07048e5f-666c-4655-9c10-b4f19e781685 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:12.697075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:12.698039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:12.698345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:12.698619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] PATH_INFO: `/v3/projects/769cca51265f4a1b9d6047809a06a487` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:12.699810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769cca51265f4a1b9d6047809a06a487)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:12.701807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:12.809817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769cca51265f4a1b9d6047809a06a487', 'request_id': 'req-828d41f1-f580-423c-b808-6ac9891e0eb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.816982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769cca51265f4a1b9d6047809a06a487', 'request_id': 'req-828d41f1-f580-423c-b808-6ac9891e0eb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.891055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769cca51265f4a1b9d6047809a06a487', 'request_id': 'req-828d41f1-f580-423c-b808-6ac9891e0eb5'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.903932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] The token cache is being invalidate because project 769cca51265f4a1b9d6047809a06a487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:12.904290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.904956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-828d41f1-f580-423c-b808-6ac9891e0eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:12.906519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1230/2428] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:12 2022] DELETE /identity/v3/projects/769cca51265f4a1b9d6047809a06a487 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:12.912207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:12.912732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:12.912829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:12.930427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:12.931666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:12.942736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:12.997108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1199/2429] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:13.012812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.013110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.013433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.033249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:13.035524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:13.043851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1231/2430] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:13.048109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.058275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae321e-93df-44a4-adb6-b078ed2855ee None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.063368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.064055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.064342 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.064617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.065416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:13.065732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.067370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.081369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1200/2431] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:35:13.085230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.094699 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f9640-64e2-4574-812c-e0bd7ca83bc2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.100513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.101220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.101560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.101827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.103677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.104646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.118685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:13.200669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1232/2432] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:13.202873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.213504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b504f6-abf3-41c0-86e7-b647d9fa7084 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.290764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.291446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.291731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.292010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.292701 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.294685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.377505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1201/2433] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:13.382290 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.393077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7e62f-1423-4aa7-8cb0-f4724d0d9ea1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.400074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.400853 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.401131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.401441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.402176 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.403807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.415261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1233/2434] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:13.419638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.429890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365640d-8069-417e-9310-05d08c5cf70a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.434508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.435156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.439087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.439087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.439087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.439087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a109a-ad07-4180-bc4b-6dcb96c8f433 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.448740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1202/2435] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:13.453652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.465728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d74b2-f58c-401a-90f0-21f94309aeb7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.470534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.471393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.472080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.472497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] PATH_INFO: `/v3/projects/95ac522b444447fb8f5389f7b9d5411b/users/a6312889631f4eef94e7ac3aa0b08ee5/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.488337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ac522b444447fb8f5389f7b9d5411b, user_id=a6312889631f4eef94e7ac3aa0b08ee5, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.491077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.617558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1234/2436] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:13 2022] PUT /identity/v3/projects/95ac522b444447fb8f5389f7b9d5411b/users/a6312889631f4eef94e7ac3aa0b08ee5/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:13.629868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.630162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.630445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.659928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] MFA Rules not processed for user `a6312889631f4eef94e7ac3aa0b08ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:13.661377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:13.744562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1203/2437] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:13.749402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.814440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03c2e7-a43f-49b0-ac34-69ccaf1023a0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.820169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:13.821218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:13.821383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:13.821560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:13.822260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:13.824199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:13.945635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1235/2438] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:13.949639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:13.961639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b6714-9907-4abe-b3e3-04d983d4a98a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:13.971336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:14.025012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.025730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.026021 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.026296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.027371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.029496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.217160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1204/2439] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:13 2022] POST /identity/v3/users => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:14.221333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.232901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d5ac1-68c5-4d6f-9c5f-b1ca8a51d535 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.238301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.238962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.239351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.240709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.240709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.242165 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.253675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1236/2440] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:14.259928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.270710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9cb87-e92c-428c-8ddd-4a2cea7502e1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.275758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.276686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.276974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.277254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] PATH_INFO: `/v3/projects/3b1747c00e4c4f8f88ad6be52633204a/users/e45399f9742943e1a44051d514d68f6a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.296233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1747c00e4c4f8f88ad6be52633204a, user_id=e45399f9742943e1a44051d514d68f6a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.299017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.402292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1205/2441] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:14 2022] PUT /identity/v3/projects/3b1747c00e4c4f8f88ad6be52633204a/users/e45399f9742943e1a44051d514d68f6a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:14.406454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.417359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283e015-8245-407b-9147-84f42fa28479 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.474899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.475486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.475773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.476054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.476792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.478449 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33961f7-d461-4997-97b9-ae1216db2ce0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.489104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1237/2442] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:35:14.493330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.504388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d692d04-303f-4bb4-bb6f-93204d6bbd3b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.509232 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.510163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.510448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.510730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] PATH_INFO: `/v3/projects/3b1747c00e4c4f8f88ad6be52633204a/users/e45399f9742943e1a44051d514d68f6a/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.512799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1747c00e4c4f8f88ad6be52633204a, user_id=e45399f9742943e1a44051d514d68f6a, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.515452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.527758 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1206/2443] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:14 2022] PUT /identity/v3/projects/3b1747c00e4c4f8f88ad6be52633204a/users/e45399f9742943e1a44051d514d68f6a/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:14.538869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.539223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.539504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.566594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] MFA Rules not processed for user `e45399f9742943e1a44051d514d68f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:14.567855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:14.585779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:14.637491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1238/2444] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:14.648835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.650754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:14.714033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8aee8-778f-4073-9298-f6e93fb4203f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.717766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.718423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.718718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.718987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.731466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.732993 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.743193 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1207/2445] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:14.843249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:14.853368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b268834-1b4e-4ba9-b805-1466831ce0ef None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:14.857221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:14.864070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:14.864070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:14.864070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:14.917582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:14.919044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e403dc-1e18-4a6c-82f8-c4215ef2e674 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:14.927996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1239/2446] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:18.352080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:18.421630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f996eac-7165-4050-9fc5-b3b7b206560b service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:18.426808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:18.427704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:18.428016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:18.428294 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] PATH_INFO: `/v3/users/a6312889631f4eef94e7ac3aa0b08ee5` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:18.429714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6312889631f4eef94e7ac3aa0b08ee5)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:18.431745 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:18.438163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:18.649583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6312889631f4eef94e7ac3aa0b08ee5'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:18.668492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6312889631f4eef94e7ac3aa0b08ee5'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:18.793959 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6312889631f4eef94e7ac3aa0b08ee5'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:18.794788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6312889631f4eef94e7ac3aa0b08ee5'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:18.795522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6312889631f4eef94e7ac3aa0b08ee5'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:18.809096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1208/2447] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:18 2022] DELETE /identity/v3/users/a6312889631f4eef94e7ac3aa0b08ee5 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:18.813582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:18.814086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:18.814322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:18.848657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:18.848657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:18.911426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1240/2448] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:18.921555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:18.923290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:18.992602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731e355-6180-496e-8ef0-5194fdce58d4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:18.997157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:19.001788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:19.001788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:19.001788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:19.011390 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:19.012889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:19.024151 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1209/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:19.556905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:19.558755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:19.582168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1538af-b6e8-4eb9-8c18-f4fbf567b457 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:19.594482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:19.595380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:19.595665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:19.595949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] PATH_INFO: `/v3/projects/95ac522b444447fb8f5389f7b9d5411b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:19.597486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ac522b444447fb8f5389f7b9d5411b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:19.600577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:19.764223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:19.765673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ac522b444447fb8f5389f7b9d5411b', 'request_id': 'req-7d922056-0e56-4b00-b0f7-9e41c121aaa6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:19.771573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ac522b444447fb8f5389f7b9d5411b', 'request_id': 'req-7d922056-0e56-4b00-b0f7-9e41c121aaa6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:19.863406 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ac522b444447fb8f5389f7b9d5411b', 'request_id': 'req-7d922056-0e56-4b00-b0f7-9e41c121aaa6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:19.876213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] The token cache is being invalidate because project 95ac522b444447fb8f5389f7b9d5411b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:19.876660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:19.877429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:19.880512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1241/2450] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:19 2022] DELETE /identity/v3/projects/95ac522b444447fb8f5389f7b9d5411b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:19.884308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:19.894999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04bb6-2b09-479e-9b0f-5f65db6f628a service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:19.958037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:19.958918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:19.959201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:19.959478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] PATH_INFO: `/v3/users/e45399f9742943e1a44051d514d68f6a` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:19.960667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45399f9742943e1a44051d514d68f6a)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:19.962658 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:20.131470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:20.144900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45399f9742943e1a44051d514d68f6a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.156784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45399f9742943e1a44051d514d68f6a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.245167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45399f9742943e1a44051d514d68f6a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.245846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45399f9742943e1a44051d514d68f6a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.246458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9b3c2fb8-8ad2-4937-9af2-d1dd12af38f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45399f9742943e1a44051d514d68f6a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.260538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1210/2451] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:19 2022] DELETE /identity/v3/users/e45399f9742943e1a44051d514d68f6a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:20.680935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:20.682942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:20.716897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d922056-0e56-4b00-b0f7-9e41c121aaa6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:20.763317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:20.764328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:20.764621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:20.764893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] PATH_INFO: `/v3/projects/3b1747c00e4c4f8f88ad6be52633204a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:20.766087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1747c00e4c4f8f88ad6be52633204a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:20.768455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:20.897758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1747c00e4c4f8f88ad6be52633204a', 'request_id': 'req-2beb7d5f-4213-41c9-aade-83ef34d9f77e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.903509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1747c00e4c4f8f88ad6be52633204a', 'request_id': 'req-2beb7d5f-4213-41c9-aade-83ef34d9f77e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:20.999078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1747c00e4c4f8f88ad6be52633204a', 'request_id': 'req-2beb7d5f-4213-41c9-aade-83ef34d9f77e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:21.007577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] The token cache is being invalidate because project 3b1747c00e4c4f8f88ad6be52633204a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:21.007862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:21.008547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-2beb7d5f-4213-41c9-aade-83ef34d9f77e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:21.010156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1242/2452] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:20 2022] DELETE /identity/v3/projects/3b1747c00e4c4f8f88ad6be52633204a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:21.016047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.016696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.016696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.043377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:21.044635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:21.115754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1211/2453] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:21.130346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.130346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.130346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.145587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:21.147186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:21.154975 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1243/2454] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:21.162274 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.169568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297218e-aeec-45a5-9ed1-4dfc1ad92fd7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.174241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.175729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.175729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.176045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.176723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:21.177034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.178761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.191976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1212/2455] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:35:21.196395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.206192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475b95b-23a2-4db7-999e-4292eb3a192c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.210985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.211643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.211927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.212204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.212905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.214884 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.305666 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1244/2456] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:21.309915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.320509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442afd28-1f24-4c38-bdcb-5b3351df5c3e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.382109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.382790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.383080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.383388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.384115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.386138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.504582 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1213/2457] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:21.509409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.524353 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82bcb6-48e6-40ae-a0e3-4ff6d2f7bbac admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.531334 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.532390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.532686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.533083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.534050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.535985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.552122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1245/2458] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:21.556029 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.567515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fba49-5235-4fc6-b2e5-318dd43300f8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.573527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.574324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.580320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.580320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.580320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.580320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f149b-5418-44ca-8525-d038453df4c4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.591452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1214/2459] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:21.594082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.604835 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9526-aaea-4c4b-a5e9-eff3e19c4338 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.609775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.610798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.611156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.611516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] PATH_INFO: `/v3/projects/8ef0a7107cfa41e493878a194a2b6d54/users/d73e3d45dcb842c5bd6af31e89cbdded/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.624629 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.625186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.625683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.629935 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef0a7107cfa41e493878a194a2b6d54, user_id=d73e3d45dcb842c5bd6af31e89cbdded, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.635014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aa152-78db-4b8e-81e1-8e7db9f698ab admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:21.652195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:21.653600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:21.668398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:21.723992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1215/2460] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:21.736486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.741801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:21.793383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1246/2461] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:21 2022] PUT /identity/v3/projects/8ef0a7107cfa41e493878a194a2b6d54/users/d73e3d45dcb842c5bd6af31e89cbdded/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:35:21.801059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.801486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.801706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.807499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:21.830873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] MFA Rules not processed for user `d73e3d45dcb842c5bd6af31e89cbdded`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:21.831407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542a6d7-b293-48cd-b632-44d69109042a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:21.832282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:21.916187 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:21.917644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:21.917933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:21.918267 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:21.924611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1247/2462] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:21.930623 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:21.997839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:21.999800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.011203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1216/2463] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:35:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:35:22.021751 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34829f5-e760-481f-8e6d-5e4b13a4f82a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.029864 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.030554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.030838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.031119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.031838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.033863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.184424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1248/2464] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:21 2022] POST /identity/v3/projects => generated 385 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:22.192105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:22.203854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a24a5e-0555-4330-bb64-545b262b0804 service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.275569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.276570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.277891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.278262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.279050 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.281369 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.336844 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:22.420266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1217/2465] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:22 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:22.430141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:22.458524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c2cca-1f44-470d-b21d-27332222a6ee admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.478031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.479735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.482141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.484125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.485984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.494743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.524972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1249/2466] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:22.528728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:22.557430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389d94-a798-4987-b68b-447ebf1abe60 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.566243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.567878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.568297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.568754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] PATH_INFO: `/v3/projects/9fff09dc34574ff38fa359717f33475e/users/e266b4094ac646e388c8e30e32a01421/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.604457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fff09dc34574ff38fa359717f33475e, user_id=e266b4094ac646e388c8e30e32a01421, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.610188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.716420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1218/2467] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:22 2022] PUT /identity/v3/projects/9fff09dc34574ff38fa359717f33475e/users/e266b4094ac646e388c8e30e32a01421/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:35:22.723731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:22.738312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74985c-7ec8-4058-8c14-5e4c9b791537 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.844744 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.845838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.846337 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.846733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.847624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.849574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e326c9b-2592-41ef-b7d9-daae3255e21b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.870083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1250/2468] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:22.871033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:22.887363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312019-d3bb-4144-b6da-87f8709d7c0a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:22.892696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:22.895360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.896515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.896515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] PATH_INFO: `/v3/projects/9fff09dc34574ff38fa359717f33475e/users/e266b4094ac646e388c8e30e32a01421/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.913684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fff09dc34574ff38fa359717f33475e, user_id=e266b4094ac646e388c8e30e32a01421, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:22.920518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:22.952600 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1219/2469] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:22 2022] PUT /identity/v3/projects/9fff09dc34574ff38fa359717f33475e/users/e266b4094ac646e388c8e30e32a01421/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:22.967155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:22.967575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:22.967726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:22.974964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:22.997912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] MFA Rules not processed for user `e266b4094ac646e388c8e30e32a01421`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:23.000196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:23.108927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1251/2470] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:23.125985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:23.128763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:23.243631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64868313-4879-493a-9003-cee7fab8bb53 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:23.257914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:23.258612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:23.258899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:23.259207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:23.408311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:23.411581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6c3b4-187b-4363-88fa-e5b67ab8f59b service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:23.447767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1220/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:29.395480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:29.397169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:29.411852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e0c1-7b5d-40fc-8f44-8fbf3a8dbd26 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:29.415612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:29.416268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:29.416559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:29.416850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:29.429640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:29.431114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f9aa9-f5aa-49e0-ba8a-1d3589a0f20a service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:29.447468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1252/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:33.060439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.060995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.061095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] PATH_INFO: `/` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.062451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1221/2473] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:35:33.068420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59a017-e349-4290-b5b1-36e48165a46f None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.070336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59a017-e349-4290-b5b1-36e48165a46f None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.070336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59a017-e349-4290-b5b1-36e48165a46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.089763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6c59a017-e349-4290-b5b1-36e48165a46f None None] MFA Rules not processed for user `5e62e836ab6e4f42814b3151bfe686d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:33.091263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6c59a017-e349-4290-b5b1-36e48165a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:33.147912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1253/2474] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:33.159889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:33.164076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:33.176974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b269c7-8f89-4833-a535-f35872ea8432 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:33.180523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:33.181235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.181557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.181863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.186825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:33.189204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:33.190665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107d472-cf5b-4e39-9d8b-df2e4f33003b service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:33.199101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1222/2475] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:33.284603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a379-eec0-4c4f-be31-7cb1f19c7ef5 None None] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.284901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a379-eec0-4c4f-be31-7cb1f19c7ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.285252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61a379-eec0-4c4f-be31-7cb1f19c7ef5 None None] PATH_INFO: `/` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.286715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1254/2476] 10.222.0.42 () {54 vars in 921 bytes} [Fri May 27 16:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 16:35:33.292080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.292372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.292650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.310860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] MFA Rules not processed for user `cb545deab49645c5947c3eacd618dc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:33.312062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:33.385982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1223/2477] 10.222.0.42 () {58 vars in 1018 bytes} [Fri May 27 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:33.397929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61a379-eec0-4c4f-be31-7cb1f19c7ef5 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:33.408755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61a379-eec0-4c4f-be31-7cb1f19c7ef5 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:33.416147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:33.416147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:33.416147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:33.416147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:33.424199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:33.425705 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d7484-273b-40af-96fe-c30727b9ed7d service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:33.436799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1255/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:36.998297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:37.066650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e4c4-0082-46d9-a3f0-cb8554bcb2af None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:37.071872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:37.072751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:37.073038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:37.073319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] PATH_INFO: `/v3/users/d73e3d45dcb842c5bd6af31e89cbdded` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:37.074588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73e3d45dcb842c5bd6af31e89cbdded)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:37.076551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:37.273072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73e3d45dcb842c5bd6af31e89cbdded'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:37.286997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73e3d45dcb842c5bd6af31e89cbdded'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:37.382892 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73e3d45dcb842c5bd6af31e89cbdded'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:37.383766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73e3d45dcb842c5bd6af31e89cbdded'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:37.384526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73e3d45dcb842c5bd6af31e89cbdded'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:37.399177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1224/2479] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:36 2022] DELETE /identity/v3/users/d73e3d45dcb842c5bd6af31e89cbdded => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:37.404169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a31eb9-9ae6-4287-9d42-7c4754ed5e6d None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:37.404667 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a31eb9-9ae6-4287-9d42-7c4754ed5e6d None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:37.404777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a31eb9-9ae6-4287-9d42-7c4754ed5e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:37.423608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b1a31eb9-9ae6-4287-9d42-7c4754ed5e6d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:37.424805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b1a31eb9-9ae6-4287-9d42-7c4754ed5e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:37.482809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1256/2480] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:37.492456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:37.493887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:37.504394 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:37.553056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b470a8-ed9c-44b2-80bf-dc74c8e463f3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:37.557253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:37.557958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:37.558261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:37.558536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:37.567500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:37.568925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:37.578290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1225/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:38.065179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:38.065512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:38.065793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:38.084073 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:38.085431 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:38.090804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:38.142830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1257/2482] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:35:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:38.155518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:38.156651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:38.157117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:38.158654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:38.178384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f6500-b8f4-4e74-b040-745fbaaf9d47 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:38.186876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:38.187985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:38.188155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:38.188436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] PATH_INFO: `/v3/projects/8ef0a7107cfa41e493878a194a2b6d54` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:38.189771 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef0a7107cfa41e493878a194a2b6d54)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:38.191799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:38.235418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd209c10-320f-4aa8-ae36-6388682c5f62 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:38.241803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:38.242497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:38.242814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:38.243097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:38.252304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:38.253958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:38.263910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1226/2483] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:35:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:35:38.329033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef0a7107cfa41e493878a194a2b6d54', 'request_id': 'req-178e50ee-d177-4ef6-b4c4-a8c2261d5496'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.334663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef0a7107cfa41e493878a194a2b6d54', 'request_id': 'req-178e50ee-d177-4ef6-b4c4-a8c2261d5496'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.416051 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef0a7107cfa41e493878a194a2b6d54', 'request_id': 'req-178e50ee-d177-4ef6-b4c4-a8c2261d5496'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.429474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] The token cache is being invalidate because project 8ef0a7107cfa41e493878a194a2b6d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:38.429807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.430577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.432280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1258/2484] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:38 2022] DELETE /identity/v3/projects/8ef0a7107cfa41e493878a194a2b6d54 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:35:38.436330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:38.447407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcbb55-6822-43c3-831d-21cb3508842a service swift] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:38.525482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:38.526387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:38.526675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:38.526957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] PATH_INFO: `/v3/users/e266b4094ac646e388c8e30e32a01421` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:38.528184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e266b4094ac646e388c8e30e32a01421)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:38.530416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:38.792426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:38.797746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e266b4094ac646e388c8e30e32a01421'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.809985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e266b4094ac646e388c8e30e32a01421'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.960526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e266b4094ac646e388c8e30e32a01421'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.961167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e266b4094ac646e388c8e30e32a01421'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.961836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3b5499d7-fd22-4bd1-b98f-55c549236d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e266b4094ac646e388c8e30e32a01421'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:38.979849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1227/2485] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:38 2022] DELETE /identity/v3/users/e266b4094ac646e388c8e30e32a01421 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:39.841603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:39.843069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:39.856559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e50ee-d177-4ef6-b4c4-a8c2261d5496 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:39.919608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:39.920474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:39.920759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:39.921036 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] PATH_INFO: `/v3/projects/9fff09dc34574ff38fa359717f33475e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:39.922272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fff09dc34574ff38fa359717f33475e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:39.924278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.051187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:40.058009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fff09dc34574ff38fa359717f33475e', 'request_id': 'req-d3ab39b8-1cde-48d8-83f8-0b096ff01635'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:40.063693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fff09dc34574ff38fa359717f33475e', 'request_id': 'req-d3ab39b8-1cde-48d8-83f8-0b096ff01635'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:40.166383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fff09dc34574ff38fa359717f33475e', 'request_id': 'req-d3ab39b8-1cde-48d8-83f8-0b096ff01635'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:40.179498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] The token cache is being invalidate because project 9fff09dc34574ff38fa359717f33475e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:40.179789 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:40.180447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d3ab39b8-1cde-48d8-83f8-0b096ff01635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:40.182379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1259/2486] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:39 2022] DELETE /identity/v3/projects/9fff09dc34574ff38fa359717f33475e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:40.188346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.188799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.188889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.207762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:40.209429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:40.274419 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1228/2487] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:35:40.338767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.339446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.339937 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.359423 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:40.362544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:40.369101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1260/2488] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:40.374253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.384642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437c4a8-fc12-49db-91a6-4b6df6fce478 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.389281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.397074 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.409954 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1229/2489] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:35:40.411306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.426000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b23578-8f2c-4278-b1c7-42c800541fc1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.430622 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.431283 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.431809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.432144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.433043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.435042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.519146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1261/2490] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:40 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:40.520418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.531694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ac06-b4d8-44b2-a751-1a5f757a0f04 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.594082 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.594751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.595040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.595313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.596028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.598318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.721451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1230/2491] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:40 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:40.725089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.735856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46eb6-bb99-4e61-8a0e-102b1fa4bc8b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.740803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.748918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.748918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.748918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.748918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.748918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.756079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1262/2492] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:40.760827 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.771707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc756e7-8b99-468f-8352-bf782da19243 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.776666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.777436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.777724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.778003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.778734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.780345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40ce02-403d-46d0-9000-fec12177dc11 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.791035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1231/2493] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:40.797791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:40.811855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2169ccc-bd36-4a41-b04a-5ca461faec3c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:40.816797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:40.818410 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.820527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.820832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] PATH_INFO: `/v3/projects/34c2030590b24de6ac2e49bb534c7921/users/7054bba92463449f80f50c7a59d1afdf/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:40.838818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c2030590b24de6ac2e49bb534c7921, user_id=7054bba92463449f80f50c7a59d1afdf, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:40.841730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:40.964261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1263/2494] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:40 2022] PUT /identity/v3/projects/34c2030590b24de6ac2e49bb534c7921/users/7054bba92463449f80f50c7a59d1afdf/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:40.967101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:40.967412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:40.967703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:41.000997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] MFA Rules not processed for user `7054bba92463449f80f50c7a59d1afdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:41.007236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:41.015845 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:41.090365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1232/2495] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:41.141314 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:41.143134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:41.220134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e22188-3837-4ea8-83fb-cb6d9ee472ca admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:41.226111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:41.227071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:41.227270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:41.227642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:41.238224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:41.240189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c9336-b156-484f-a709-869cdc645929 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:41.250852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1264/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:41.267828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:41.340893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5224c-e4f5-4943-8924-f8f2b2c51073 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:41.350574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:41.351823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:41.352241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:41.352439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] PATH_INFO: `/v3/users/7054bba92463449f80f50c7a59d1afdf` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:41.353742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7054bba92463449f80f50c7a59d1afdf)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:41.355900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:41.546962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7054bba92463449f80f50c7a59d1afdf'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:41.560318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7054bba92463449f80f50c7a59d1afdf'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:41.651515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7054bba92463449f80f50c7a59d1afdf'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:41.652225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7054bba92463449f80f50c7a59d1afdf'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:41.652887 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7054bba92463449f80f50c7a59d1afdf'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:41.666537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1233/2497] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:41 2022] DELETE /identity/v3/users/7054bba92463449f80f50c7a59d1afdf => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:41.671220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:41.671532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:41.671808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:41.690474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:41.691698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:41.755450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1265/2498] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:41.766195 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:41.767635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:41.804569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:41.828214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad4efe-7c8f-406b-bbf4-808e7bf492f8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:41.832075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:41.832728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:41.833026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:41.833303 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:41.845756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:41.848869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:41.858727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1234/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:42.456943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:42.459306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:42.482263 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d18ea9-f8f5-41f7-b585-22a375a7de79 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:42.488844 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:42.490717 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:42.491066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:42.491489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] PATH_INFO: `/v3/projects/34c2030590b24de6ac2e49bb534c7921` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:42.492814 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c2030590b24de6ac2e49bb534c7921)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:42.494913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:42.497542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:42.661080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c2030590b24de6ac2e49bb534c7921', 'request_id': 'req-83cd4364-2fc9-42c2-b83e-c006d210d4cf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:42.667695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c2030590b24de6ac2e49bb534c7921', 'request_id': 'req-83cd4364-2fc9-42c2-b83e-c006d210d4cf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:42.778869 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c2030590b24de6ac2e49bb534c7921', 'request_id': 'req-83cd4364-2fc9-42c2-b83e-c006d210d4cf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:42.801818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] The token cache is being invalidate because project 34c2030590b24de6ac2e49bb534c7921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:42.803014 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:42.804102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:42.806547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1266/2500] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:42 2022] DELETE /identity/v3/projects/34c2030590b24de6ac2e49bb534c7921 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:43.973032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:43.974927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:43.987484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f2786-626b-4a21-9e91-96ce4da513dd service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:44.044437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:44.045096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:44.045418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:44.045697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:44.047736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:44.116382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:44.117987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:44.128616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1235/2501] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:47.831720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:47.833049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:47.847639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd4364-2fc9-42c2-b83e-c006d210d4cf admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:47.852824 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:47.853719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:47.854002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:47.854279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] PATH_INFO: `/v3/users/87d41932bfa94f369103ed29d29940b3` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:47.855697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d41932bfa94f369103ed29d29940b3)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:47.857680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:48.059340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d41932bfa94f369103ed29d29940b3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.060082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d41932bfa94f369103ed29d29940b3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.132545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d41932bfa94f369103ed29d29940b3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.133098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d41932bfa94f369103ed29d29940b3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.145572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c5a3d5e4-851c-48ed-9925-a5b263831f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d41932bfa94f369103ed29d29940b3'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.158747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1267/2502] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:47 2022] DELETE /identity/v3/users/87d41932bfa94f369103ed29d29940b3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:48.407262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:48.419443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bc73c-889f-428b-a8ef-7a7bf4369c90 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:48.486485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:48.487332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:48.487613 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:48.487895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] PATH_INFO: `/v3/projects/3fa953c1f6d240098f164785b8ccc5b4` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:48.489081 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa953c1f6d240098f164785b8ccc5b4)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:48.492003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:48.596583 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:48.612615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa953c1f6d240098f164785b8ccc5b4', 'request_id': 'req-8f3f4c14-3f18-42a0-b28c-5b75552bb738'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.691603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa953c1f6d240098f164785b8ccc5b4', 'request_id': 'req-8f3f4c14-3f18-42a0-b28c-5b75552bb738'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.698461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa953c1f6d240098f164785b8ccc5b4', 'request_id': 'req-8f3f4c14-3f18-42a0-b28c-5b75552bb738'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.709997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] The token cache is being invalidate because project 3fa953c1f6d240098f164785b8ccc5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:48.710292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.711015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8f3f4c14-3f18-42a0-b28c-5b75552bb738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:48.712739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1236/2503] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:48 2022] DELETE /identity/v3/projects/3fa953c1f6d240098f164785b8ccc5b4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:48.717859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:48.718160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:48.718454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:48.736645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:48.737989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:48.800428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1268/2504] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:48.808806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:48.809128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:48.809457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:48.828822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:48.830128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:48.838203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1237/2505] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:48.848743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:48.854989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:48.858785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e4a8-6431-49a2-963f-744decbf2c70 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:48.863536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:48.864193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:48.864478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:48.865695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:48.866166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:48.866481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:48.868095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:48.879902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1269/2506] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:35:48.893078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:48.902591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a7aa7-61ec-4e26-a326-93bb3fe11851 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:48.907188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:48.907832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:48.908118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:48.908398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:48.909092 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:48.911040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.002145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1238/2507] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:48 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:49.006504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:49.016603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c1a90-0ad6-43fe-8bff-c8832fcf00fe admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:49.074144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:49.074812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.075100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.075382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.076068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:49.078041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.160455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1270/2508] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:49 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:49.164635 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:49.175117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ca4e5-5d17-4620-9f1a-6804340dc888 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:49.180481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:49.187891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.187891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.187891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.187891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:49.187891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.196219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1239/2509] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:49.200503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:49.210903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e88b52-3beb-40eb-86a1-2d027b23d639 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:49.215654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:49.216305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.222309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.222309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.222309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:49.222309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bf7f9-f2b4-431e-9f12-62271d441ae9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.229828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1271/2510] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:49.234116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:49.244980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761083-1a22-43c2-ad48-4776a578f30a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:49.250023 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:49.250931 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.251215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.256712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] PATH_INFO: `/v3/projects/04cb1199226645fba8c2bc7dca36e038/users/9aa0097e93bf4b8d8da23234fae0e475/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.266898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04cb1199226645fba8c2bc7dca36e038, user_id=9aa0097e93bf4b8d8da23234fae0e475, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:49.269657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.355237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1240/2511] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:49 2022] PUT /identity/v3/projects/04cb1199226645fba8c2bc7dca36e038/users/9aa0097e93bf4b8d8da23234fae0e475/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:49.366086 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.366387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.366670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.393209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] MFA Rules not processed for user `9aa0097e93bf4b8d8da23234fae0e475`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:49.394457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:49.459254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1272/2512] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:49.470627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:49.472379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:49.558736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b59b9-3e39-401f-99ee-23ad33c5da1e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:49.568966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:49.571349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:49.571642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:49.571918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:49.580539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:49.582358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a143f8-7d1e-4d5d-a012-a8682363f4d0 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:49.593031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1241/2513] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:53.150455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:53.214673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb063-6fdc-47f3-9a0c-aa17297357f7 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:53.219191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:53.220082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:53.220374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:53.220657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] PATH_INFO: `/v3/users/9aa0097e93bf4b8d8da23234fae0e475` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:53.221899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa0097e93bf4b8d8da23234fae0e475)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:53.223839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:53.226098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:53.463478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa0097e93bf4b8d8da23234fae0e475'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:53.464150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa0097e93bf4b8d8da23234fae0e475'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:53.549815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa0097e93bf4b8d8da23234fae0e475'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:53.551360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa0097e93bf4b8d8da23234fae0e475'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:53.565557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa0097e93bf4b8d8da23234fae0e475'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:53.579829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1273/2514] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:35:53 2022] DELETE /identity/v3/users/9aa0097e93bf4b8d8da23234fae0e475 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:53.584447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:53.587031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:53.587031 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:53.604590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:53.605894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:53.641776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:53.671101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1242/2515] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:53.681635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:53.684578 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:53.740416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633beac-9ae1-4148-adbb-d3f78094798f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:53.745081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:53.745813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:53.750741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:53.750741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:53.758175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:53.759722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce37e1-ca75-4f19-b0cd-ec47058ad0d9 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:53.771429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1274/2516] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:35:54.442280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:54.444287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:54.458210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5130d8a-ca5d-446e-acab-7de500736165 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:54.463733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:54.464715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:54.465072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:54.465550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] PATH_INFO: `/v3/projects/04cb1199226645fba8c2bc7dca36e038` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:54.466888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=04cb1199226645fba8c2bc7dca36e038)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:54.469179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:54.569240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04cb1199226645fba8c2bc7dca36e038', 'request_id': 'req-ed873623-9a36-4f15-9789-a2c54bfb4bce'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:54.628727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04cb1199226645fba8c2bc7dca36e038', 'request_id': 'req-ed873623-9a36-4f15-9789-a2c54bfb4bce'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:54.634961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04cb1199226645fba8c2bc7dca36e038', 'request_id': 'req-ed873623-9a36-4f15-9789-a2c54bfb4bce'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:54.646746 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] The token cache is being invalidate because project 04cb1199226645fba8c2bc7dca36e038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:35:54.647039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:54.647773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed873623-9a36-4f15-9789-a2c54bfb4bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:35:54.649452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1243/2517] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:35:54 2022] DELETE /identity/v3/projects/04cb1199226645fba8c2bc7dca36e038 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:35:54.654540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:54.655045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:54.655194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:54.674847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:54.676071 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:54.736077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1275/2518] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:54.744404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:54.744813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:54.745324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:54.765100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:54.766365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:54.776102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1244/2519] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:54.780316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:54.791259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90407db-b67d-46c5-b208-7ad3fe98d5a2 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:54.795979 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:54.809544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:54.811841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1276/2520] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:35:54.816071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:54.826227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b36524-7832-4158-8a07-a7245b2c908e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:54.831200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:54.831859 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:54.832172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:54.832455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:54.833165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:54.835973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:54.985415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1245/2521] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:35:54 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:35:54.989715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:55.000812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19de2-6385-48c7-a2f8-f16042f4c648 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:55.006589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:55.066066 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:55.066704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.066994 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.067417 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.069019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:55.071074 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:55.184574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1277/2522] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:35:54 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:35:55.188912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:55.201299 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8675b18-9d2c-4058-aa7f-9f4c8f64f85d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:55.209457 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:55.210463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.210757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.211349 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.212093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:55.213754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:55.224721 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1246/2523] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:55.228965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:55.239053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e0a1-62e5-4922-8e5a-dc97fa9be8b5 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:55.244200 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:55.244907 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.245195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.245513 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.246234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:55.247862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548072-cc41-4cee-a25d-765110a220d9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:55.259186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1278/2524] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:35:55.263558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:55.275951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb8613-20d9-457d-b23c-efe177fb5d5c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:55.280850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:55.281767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.282053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.282327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] PATH_INFO: `/v3/projects/aef72cb91d9240c9a1676c0a5e70e744/users/83bd3331cc974dec8e242db56a21f34b/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.298228 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef72cb91d9240c9a1676c0a5e70e744, user_id=83bd3331cc974dec8e242db56a21f34b, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:55.301076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:55.388974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1247/2525] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:35:55 2022] PUT /identity/v3/projects/aef72cb91d9240c9a1676c0a5e70e744/users/83bd3331cc974dec8e242db56a21f34b/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:35:55.400326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.400620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.400904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.433495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] MFA Rules not processed for user `83bd3331cc974dec8e242db56a21f34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:35:55.435412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:55.506960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1279/2526] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:35:55.518742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:35:55.520516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:35:55.588493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b3375-c02f-42c2-a069-138d1c5064fa admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:35:55.592436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:35:55.593085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:35:55.593400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:35:55.593817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:35:55.602908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:35:55.604975 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:35:55.612858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:35:55.619568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1248/2527] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:35:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:04.851318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:04.852833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:04.910981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd4dda-25ba-4a05-a657-a2324c01bc83 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:04.914654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:04.915297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:04.915584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:04.915865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:04.917670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:04.950518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:04.984236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:04.986000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecb62b-9d73-4f30-b6ef-daf677a29f3d service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:04.997147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1280/2528] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:05.860262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:05.871401 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ba99-a20b-4abc-b6b9-792f6a15e9be service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:05.876482 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:05.877404 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:05.877820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:05.878027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] PATH_INFO: `/v3/users/83bd3331cc974dec8e242db56a21f34b` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:05.879264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bd3331cc974dec8e242db56a21f34b)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:05.882067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:06.168523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bd3331cc974dec8e242db56a21f34b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:06.182358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bd3331cc974dec8e242db56a21f34b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:06.268869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bd3331cc974dec8e242db56a21f34b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:06.269470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bd3331cc974dec8e242db56a21f34b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:06.270117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bd3331cc974dec8e242db56a21f34b'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:06.293669 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1249/2529] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:05 2022] DELETE /identity/v3/users/83bd3331cc974dec8e242db56a21f34b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:06.295684 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:06.296156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:06.296488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:06.328862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:06.330301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:06.402991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1281/2530] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:06.412855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:06.414960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:06.478744 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634440ff-4883-4293-8e1a-4212e0e3f45b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:06.482424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:06.483085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:06.483372 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:06.483651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:06.494708 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:06.496165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557abad8-f75d-4f34-af24-0a02e3f738e9 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:06.507860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1250/2531] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:07.144969 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:07.147326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:07.161371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793900e0-c310-4383-8ea9-13526b3b6b28 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:07.166077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:07.167035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.167388 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.167732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] PATH_INFO: `/v3/projects/aef72cb91d9240c9a1676c0a5e70e744` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.168999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef72cb91d9240c9a1676c0a5e70e744)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:07.171072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:07.347560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef72cb91d9240c9a1676c0a5e70e744', 'request_id': 'req-c16df502-2470-43c1-9ad1-ceabd4e051fd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:07.359433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef72cb91d9240c9a1676c0a5e70e744', 'request_id': 'req-c16df502-2470-43c1-9ad1-ceabd4e051fd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:07.490618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef72cb91d9240c9a1676c0a5e70e744', 'request_id': 'req-c16df502-2470-43c1-9ad1-ceabd4e051fd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:07.503898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] The token cache is being invalidate because project aef72cb91d9240c9a1676c0a5e70e744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:07.504291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:07.505693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c16df502-2470-43c1-9ad1-ceabd4e051fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:07.507402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1282/2532] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:07 2022] DELETE /identity/v3/projects/aef72cb91d9240c9a1676c0a5e70e744 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:07.512210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.512718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.512851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.531093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:07.532309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:07.597861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1251/2533] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:07.606861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186350-f745-42d1-9e50-3da42415668c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.607195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186350-f745-42d1-9e50-3da42415668c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.607487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1186350-f745-42d1-9e50-3da42415668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.626822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d1186350-f745-42d1-9e50-3da42415668c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:07.628075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d1186350-f745-42d1-9e50-3da42415668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:07.638168 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1283/2534] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:07.642329 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:07.653686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7cab-0358-4051-b928-f763a3a79e70 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:07.658823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:07.664376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:07.675523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1252/2535] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:07.680080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1186350-f745-42d1-9e50-3da42415668c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:07.690062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1186350-f745-42d1-9e50-3da42415668c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:07.696640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:07.697316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.697637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.697918 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.698658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:07.700866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:07.842575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1284/2536] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:07 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:07.851999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:07.858136 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:07.864532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbef09b-02d4-46a2-a86d-b2a51b13f0f1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:07.934678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:07.935377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:07.935720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:07.936415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:07.936683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:07.938654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.087861 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1253/2537] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:07 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:08.092536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:08.104371 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71710-4df4-4ab4-bd7d-4c0c8f253532 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:08.118763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:08.118763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.118763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.118763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.119301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:08.119301 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.133380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1285/2538] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:08.136066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:08.147812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a4c7-486f-4199-ae85-b31997330c6f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:08.163741 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:08.164447 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.164736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.165019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.165787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:08.174060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee499bf-0cb9-4205-93e9-cc4cf8746441 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.189140 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1254/2539] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:08.194202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:08.207594 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e728c-e9b5-45ef-bb32-d6e2b5192439 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:08.225218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:08.225218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.225218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.225218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] PATH_INFO: `/v3/projects/14f881b6734c484f88b81d4b27eab07c/users/8e1770a6556b48418358e7b1b060f431/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.225802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:08.230262 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f881b6734c484f88b81d4b27eab07c, user_id=8e1770a6556b48418358e7b1b060f431, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:08.233088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.350115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1286/2540] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:08 2022] PUT /identity/v3/projects/14f881b6734c484f88b81d4b27eab07c/users/8e1770a6556b48418358e7b1b060f431/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:08.365774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.366065 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.366346 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.401248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] MFA Rules not processed for user `8e1770a6556b48418358e7b1b060f431`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:08.402840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:08.527466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1255/2541] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:08.528628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:08.629282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3836d-84c5-4885-8a11-d63ab5c0ab49 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:08.636640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:08.637386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.637936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.638769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.639838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:08.643304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.776058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1287/2542] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:08 2022] POST /identity/v3/projects => generated 371 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:08.777458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:08.788625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1fee-883b-46b6-86c6-e35ec7f7f879 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:08.852957 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:08.853722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:08.854034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:08.854321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:08.855264 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:08.857568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:08.989189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:09.004048 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1256/2543] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:08 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:09.009142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.022235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73232-3b3e-49e0-b949-e3c4553bf797 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.027427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.028109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.028395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.028720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.029501 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.031207 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.043424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1288/2544] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:09.047471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.054351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.056707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:09.059454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0821454-44c6-4a33-8412-3d8695f65ab1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.065893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.068243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.068375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.068664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] PATH_INFO: `/v3/projects/f485a82d014b45ca9fa15acb31872b68/users/5a5d33c5e167457184f6b2d479161c38/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.088667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f485a82d014b45ca9fa15acb31872b68, user_id=5a5d33c5e167457184f6b2d479161c38, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.091566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.140158 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d471e-b5d4-4c91-ab47-d569f7f3d0e6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.146182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.146925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.147211 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.147486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.149349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:09.161194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.163435 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.173574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1289/2545] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:09.220398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1257/2546] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:09 2022] PUT /identity/v3/projects/f485a82d014b45ca9fa15acb31872b68/users/5a5d33c5e167457184f6b2d479161c38/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:09.227803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.227941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.228235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:09.238130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6fd54-9de9-4a7a-8fba-1a06399786c9 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.290873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:09.297435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e673f5-b8a4-4a43-b0b6-ce5e7fb1aa4c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.301149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.301792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.302627 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.302880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.303000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.303619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.303784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.304054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.304350 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] PATH_INFO: `/v3/users/56799d191ffe4193918fa0221c27c0e0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.305233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.305567 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56799d191ffe4193918fa0221c27c0e0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.307484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.316169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1290/2547] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:09.320252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.330731 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3a63-b0bb-495b-876b-2d70f5351e58 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.336813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.337726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.338009 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.338285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] PATH_INFO: `/v3/projects/f485a82d014b45ca9fa15acb31872b68/users/5a5d33c5e167457184f6b2d479161c38/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.340252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f485a82d014b45ca9fa15acb31872b68, user_id=5a5d33c5e167457184f6b2d479161c38, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.342967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ceef-a423-4820-b747-73e65fd73d84 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.354644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1291/2548] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:09 2022] PUT /identity/v3/projects/f485a82d014b45ca9fa15acb31872b68/users/5a5d33c5e167457184f6b2d479161c38/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:09.366651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.366946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.367233 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.394373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] MFA Rules not processed for user `5a5d33c5e167457184f6b2d479161c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:09.395689 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:09.463695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1292/2549] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:09.476164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:09.477606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:09.535439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d28c1-57fc-471e-a658-e8edc43cadca None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:09.539873 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:09.540524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:09.540808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:09.541192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:09.551183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:09.552946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:09.558626 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56799d191ffe4193918fa0221c27c0e0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:09.565716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1293/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:09.572654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56799d191ffe4193918fa0221c27c0e0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:09.697295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56799d191ffe4193918fa0221c27c0e0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:09.697958 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56799d191ffe4193918fa0221c27c0e0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:09.698580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-86325035-6bdd-429c-b676-c792f081235b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56799d191ffe4193918fa0221c27c0e0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:09.712236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1258/2551] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:09 2022] DELETE /identity/v3/users/56799d191ffe4193918fa0221c27c0e0 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:09.990735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:10.075306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e91b-932b-4bd5-83d7-001cc5e0ffb3 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:10.133136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:10.133136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.136000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.136000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] PATH_INFO: `/v3/projects/c409475791e243108336a3cd2e25d3ac` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.136000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c409475791e243108336a3cd2e25d3ac)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:10.137723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:10.348101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c409475791e243108336a3cd2e25d3ac', 'request_id': 'req-601681de-5373-4356-9f50-c0482db7ff46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:10.358673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c409475791e243108336a3cd2e25d3ac', 'request_id': 'req-601681de-5373-4356-9f50-c0482db7ff46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:10.474943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c409475791e243108336a3cd2e25d3ac', 'request_id': 'req-601681de-5373-4356-9f50-c0482db7ff46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:10.487000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] The token cache is being invalidate because project c409475791e243108336a3cd2e25d3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:10.487303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:10.487986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-601681de-5373-4356-9f50-c0482db7ff46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:10.490270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1294/2552] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:09 2022] DELETE /identity/v3/projects/c409475791e243108336a3cd2e25d3ac => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:10.495320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.495644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.495964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.515829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:10.517117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:10.570863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:10.578886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1259/2553] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:10.588325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.588849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.589425 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.609156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:10.610399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:10.619201 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1295/2554] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:10.623184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:10.634750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a90b3-370f-4aeb-9b73-2d0a2fd84af2 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:10.639757 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:10.640571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.640870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.641148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.641942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:10.642893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:10.644516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:10.656933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1260/2555] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:10.662562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:10.672224 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fdcc9-935b-4672-9ded-1096ad3dd218 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:10.676865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:10.678204 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.678486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.678763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.679448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:10.681375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:10.760354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:10.762099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:10.775793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49d85e-18ad-4792-ac2d-921e05d4ffe9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:10.780611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:10.781572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.781858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.782153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] PATH_INFO: `/v3/users/8e1770a6556b48418358e7b1b060f431` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.783347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1770a6556b48418358e7b1b060f431)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:10.785279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:10.796860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1296/2556] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:10 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:10.801145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:10.863952 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26457b01-5578-4466-8aa2-b41597dd4258 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:10.869615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:10.873871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:10.873871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:10.873871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:10.873871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:10.873871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.018237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1297/2557] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:10 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:36:11.023488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.033649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072d27-b7be-4408-a925-a3e09b24f333 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.039424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.040082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.040370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.040647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.041405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.043027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.053589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1298/2558] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:11.057613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.068312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250501-beac-4908-8487-ebbb4e6c6469 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.070703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:11.075876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.076549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.076891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.080270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.080270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.080270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.086480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1770a6556b48418358e7b1b060f431'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:11.090480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1299/2559] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:11.098504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.101301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1770a6556b48418358e7b1b060f431'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:11.109777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac69af0-1486-4df0-9eb7-65ff40d80f26 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.114996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.115831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.116117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.116395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] PATH_INFO: `/v3/projects/f8222557fef247e7803f51ab81619863/users/0e4cba21dc5e4ffb9e0ebffa0bca6f82/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.132791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8222557fef247e7803f51ab81619863, user_id=0e4cba21dc5e4ffb9e0ebffa0bca6f82, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.135537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefbb9b7-7038-46ec-8f85-c25ba3f423a3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.154490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1770a6556b48418358e7b1b060f431'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:11.155208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1770a6556b48418358e7b1b060f431'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:11.155904 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-ed5225d6-a588-4392-bb56-0ed392819b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1770a6556b48418358e7b1b060f431'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:11.169817 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1261/2560] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:10 2022] DELETE /identity/v3/users/8e1770a6556b48418358e7b1b060f431 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:11.174339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566085e9-cd40-419c-b659-aadedc267597 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.174663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566085e9-cd40-419c-b659-aadedc267597 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.174948 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566085e9-cd40-419c-b659-aadedc267597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.192141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-566085e9-cd40-419c-b659-aadedc267597 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:11.193451 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-566085e9-cd40-419c-b659-aadedc267597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:11.255868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1262/2561] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:11.265880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566085e9-cd40-419c-b659-aadedc267597 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.267496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-566085e9-cd40-419c-b659-aadedc267597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:11.285747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1300/2562] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:11 2022] PUT /identity/v3/projects/f8222557fef247e7803f51ab81619863/users/0e4cba21dc5e4ffb9e0ebffa0bca6f82/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:11.296292 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.296849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.296849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.333409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] MFA Rules not processed for user `0e4cba21dc5e4ffb9e0ebffa0bca6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:11.336446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:11.338956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566085e9-cd40-419c-b659-aadedc267597 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.392561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.393121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.393442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.393723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.409134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1301/2563] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:11.414847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.416230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:11.479602 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.482087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.495227 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1263/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:11.502899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b53460-490c-4264-84aa-d3f1f3897266 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.509490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.510173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.510462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.510742 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.511447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.513468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.676448 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1302/2565] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:11 2022] POST /identity/v3/projects => generated 365 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:36:11.681009 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.690687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624026c6-6f2e-4655-ae3a-1955d0a1dcf0 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.754139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.754858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.755146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.755426 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.757183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.758734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.772660 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:11.888528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:11.894748 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1264/2566] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:11 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:11.899635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.909993 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f8d99-6d7a-46d8-9b0e-629097b031c8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.914875 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.915538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.915827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.916106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.916833 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.918497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.929936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1303/2567] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:11.934099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.945834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a3344-3afc-478b-97a7-e0e143a1e888 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.950730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.951631 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.951922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.952200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] PATH_INFO: `/v3/projects/0da774e4fea643e2aa0535e59214928d/users/fb0df214d711460a82fe2e9d66131894/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.967927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da774e4fea643e2aa0535e59214928d, user_id=fb0df214d711460a82fe2e9d66131894, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:11.970664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:11.974027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:11.975961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:11.990888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c6e3-eb0a-4c33-830d-89d371bacb1f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:11.995803 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:11.996724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:11.997710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:11.998010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] PATH_INFO: `/v3/projects/14f881b6734c484f88b81d4b27eab07c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:11.999790 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f881b6734c484f88b81d4b27eab07c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.001780 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.096478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1265/2568] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:11 2022] PUT /identity/v3/projects/0da774e4fea643e2aa0535e59214928d/users/fb0df214d711460a82fe2e9d66131894/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:12.102723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:12.136924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f881b6734c484f88b81d4b27eab07c', 'request_id': 'req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.142575 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f881b6734c484f88b81d4b27eab07c', 'request_id': 'req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.170310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee07a9-793b-4793-ad99-7446fcd9bc13 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:12.216832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f881b6734c484f88b81d4b27eab07c', 'request_id': 'req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.228109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:12.229377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.229801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.231463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.231463 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.232097 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] The token cache is being invalidate because project 14f881b6734c484f88b81d4b27eab07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:12.232443 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.232595 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.233258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.234949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1304/2569] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:11 2022] DELETE /identity/v3/projects/14f881b6734c484f88b81d4b27eab07c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:12.238836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:12.240177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.246127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1266/2570] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:12.250579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:12.252362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.253629 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d05f3-1c40-4f8b-bb3f-6aeaeda314b8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:12.258997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:12.259859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.260304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.260589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] PATH_INFO: `/v3/users/5a5d33c5e167457184f6b2d479161c38` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.261886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5d33c5e167457184f6b2d479161c38)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.263827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.265139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0552ab-85a1-46b8-8979-56b6598359fa admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:12.270168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:12.271094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.271382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.271696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] PATH_INFO: `/v3/projects/0da774e4fea643e2aa0535e59214928d/users/fb0df214d711460a82fe2e9d66131894/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.273799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da774e4fea643e2aa0535e59214928d, user_id=fb0df214d711460a82fe2e9d66131894, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.276819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18639009-54a4-4048-9f8a-c74e7e095f44 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.277567 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:12.288474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1267/2571] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:12 2022] PUT /identity/v3/projects/0da774e4fea643e2aa0535e59214928d/users/fb0df214d711460a82fe2e9d66131894/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:12.300017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.300317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.300663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.309579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:12.329144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] MFA Rules not processed for user `fb0df214d711460a82fe2e9d66131894`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:12.330483 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.402152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1268/2572] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:12.414538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:12.415945 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.466358 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5d33c5e167457184f6b2d479161c38'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.467135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5d33c5e167457184f6b2d479161c38'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.477358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c6831-2386-47e2-b232-3c138d63bda9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:12.481013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:12.481710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.481997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.482276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.484123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.496754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.498229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.508049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1269/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:12.615818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5d33c5e167457184f6b2d479161c38'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.616565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5d33c5e167457184f6b2d479161c38'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.638817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5d33c5e167457184f6b2d479161c38'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:12.645637 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:12.656986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1305/2574] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:12 2022] DELETE /identity/v3/users/5a5d33c5e167457184f6b2d479161c38 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:12.752058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:12.754263 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.765797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904688e-7e33-4962-97f7-efa1e0814d98 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:12.817978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:12.818644 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:12.819027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:12.819309 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:12.821094 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:12.887851 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:12.889481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51473f3f-e07f-4da1-9768-b2c32b4b1eff service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:12.902061 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1270/2575] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:13.128077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:13.129734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:13.202541 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6709b8-27fe-4f64-a015-77eee034d20a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:13.207109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:13.207971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.208257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.208534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] PATH_INFO: `/v3/projects/f485a82d014b45ca9fa15acb31872b68` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.210548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f485a82d014b45ca9fa15acb31872b68)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:13.212470 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:13.228838 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:13.379217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f485a82d014b45ca9fa15acb31872b68', 'request_id': 'req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:13.385584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f485a82d014b45ca9fa15acb31872b68', 'request_id': 'req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:13.482161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f485a82d014b45ca9fa15acb31872b68', 'request_id': 'req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:13.495017 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] The token cache is being invalidate because project f485a82d014b45ca9fa15acb31872b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:13.495443 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:13.496078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-d4fec4fb-368f-42bb-bd6d-d1c8e061ac6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:13.497898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1306/2576] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:13 2022] DELETE /identity/v3/projects/f485a82d014b45ca9fa15acb31872b68 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:13.503196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.503491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.503773 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.522185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:13.523490 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:13.586515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1271/2577] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:13.594241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.594538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.594827 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.616770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:13.617536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:13.627631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1307/2578] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:13.631085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:13.640910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ad574-a0b2-487d-8bfc-f47b9547f7d4 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:13.645685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:13.646374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.646680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.646960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.647692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:13.648003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:13.649667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:13.661654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1272/2579] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:36:13.671234 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:13.681481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652dd3a-17cf-4194-bd35-02dce54a3350 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:13.686143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:13.686793 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.687102 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.687378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.688104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:13.690087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:13.771407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1308/2580] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:13 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:13.775666 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:13.786252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062ec7f-37bf-4c9e-99cc-7b0c699a1998 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:13.811849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:13.844573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:13.845242 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:13.845553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:13.845833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:13.846561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:13.848489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:13.993134 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1273/2581] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:13 2022] POST /identity/v3/users => generated 279 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:13.994517 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.005149 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a69f0-c778-4ba3-9c3e-6619b20e0b71 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.010182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.020373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.020373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.020373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.020373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.020373 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.025360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1309/2582] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:14.029316 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.040300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54cb29-6c3f-4f91-aa72-52a6080bec2c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.045324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.046037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.046321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.046595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.047307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.048897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a91494-29c8-438d-b7c6-7b3a0683ac6c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.059429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1274/2583] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:14.063730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.074108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac6b3a-c506-41ef-80c4-83a835f37e09 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.078590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.079503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.079774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.080053 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] PATH_INFO: `/v3/projects/4189389e4d434140ac8b6a05de9f4bcf/users/13b1d6eadd92489ca35592665fbe71ca/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.096247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4189389e4d434140ac8b6a05de9f4bcf, user_id=13b1d6eadd92489ca35592665fbe71ca, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.099104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.196011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1310/2584] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:14 2022] PUT /identity/v3/projects/4189389e4d434140ac8b6a05de9f4bcf/users/13b1d6eadd92489ca35592665fbe71ca/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:14.206301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.206591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.206878 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.238002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] MFA Rules not processed for user `13b1d6eadd92489ca35592665fbe71ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:14.239297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:14.306189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1275/2585] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:14.310679 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.372524 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1da1e-6438-4c5a-a106-17187ffb8fa9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.378000 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.378277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.378563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.378841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.386287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.386287 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.493320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1311/2586] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:14 2022] POST /identity/v3/projects => generated 353 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:36:14.497358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.511663 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da86680-907c-4132-b0a1-a917489ce90c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.582997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.584711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.585076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.585877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.586587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.588555 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.681058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1276/2587] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:14 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:14.685128 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.696895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4d3c1-a62f-43b3-b95b-673754729fb0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.701573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.702241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.703195 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.703486 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.704222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.705945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.722487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1312/2588] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:14.724126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.734693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6e15-6660-4ffc-bbe4-4b6403bfdb83 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.739458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.740842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.741194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.741540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] PATH_INFO: `/v3/projects/0255a41891e746bf9236a8267de8609f/users/fe2d6a0022574e928061503d7bbcae64/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.758470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0255a41891e746bf9236a8267de8609f, user_id=fe2d6a0022574e928061503d7bbcae64, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.761253 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.841272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:14.849378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1277/2589] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:14 2022] PUT /identity/v3/projects/0255a41891e746bf9236a8267de8609f/users/fe2d6a0022574e928061503d7bbcae64/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:14.855049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.866377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bbc95-e7b7-460c-8522-d3ae4babc370 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.877312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:14.936374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.936374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.936374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.936374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.937558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.938219 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae094584-11be-4ca8-b382-eb77299fd6dc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.952779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1313/2590] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:14.957590 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:14.968314 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc949b-5865-4d39-992c-7c5d3e23983c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:14.973290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:14.974313 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:14.974596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:14.975223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] PATH_INFO: `/v3/projects/0255a41891e746bf9236a8267de8609f/users/fe2d6a0022574e928061503d7bbcae64/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:14.977319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0255a41891e746bf9236a8267de8609f, user_id=fe2d6a0022574e928061503d7bbcae64, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:14.980206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:14.992126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1278/2591] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:14 2022] PUT /identity/v3/projects/0255a41891e746bf9236a8267de8609f/users/fe2d6a0022574e928061503d7bbcae64/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:15.004209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:15.004611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:15.004908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:15.042514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] MFA Rules not processed for user `fe2d6a0022574e928061503d7bbcae64`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:15.043761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:15.156475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1314/2592] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:15 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:15.168080 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:15.169984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:15.238988 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d857b-0acb-4829-b854-3cacffed81e8 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:15.244221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:15.244911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:15.245203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:15.245524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:15.256027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:15.258138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:15.268960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1279/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:15 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:15.506908 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:15.516754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aac76-2ec4-4c81-9ce2-04a55afcf787 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:15.520554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:15.521216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:15.523706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:15.523706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:15.580624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:15.582276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2522d1a-f638-4b15-9331-7e49863db432 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:15.592436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1315/2594] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:16.676580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:16.679231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:16.750010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf6e5-f2f3-4bd5-869f-0de16c8932ba service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:16.754692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:16.755574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:16.755858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:16.756133 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] PATH_INFO: `/v3/users/0e4cba21dc5e4ffb9e0ebffa0bca6f82` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:16.757425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4cba21dc5e4ffb9e0ebffa0bca6f82)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:16.759384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:16.977544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4cba21dc5e4ffb9e0ebffa0bca6f82'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:16.993425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4cba21dc5e4ffb9e0ebffa0bca6f82'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:17.069005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4cba21dc5e4ffb9e0ebffa0bca6f82'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:17.069801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4cba21dc5e4ffb9e0ebffa0bca6f82'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:17.070616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4cba21dc5e4ffb9e0ebffa0bca6f82'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:17.087109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1280/2595] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:16 2022] DELETE /identity/v3/users/0e4cba21dc5e4ffb9e0ebffa0bca6f82 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:17.092045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:17.092580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:17.092580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:17.119203 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:17.121653 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:17.170674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:17.204421 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1316/2596] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:17.216119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:17.219206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:17.231810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:17.283259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80d6c-a6ab-4b2a-b04e-7b05c291fd4e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:17.286985 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:17.287752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:17.288556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:17.288840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:17.298953 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:17.300416 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:17.314198 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1281/2597] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:18.013409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:18.015022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:18.033399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a3812-0057-4ab8-9dc4-4580d1a51ba8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:18.049094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:18.050043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:18.052784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:18.053085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] PATH_INFO: `/v3/projects/f8222557fef247e7803f51ab81619863` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:18.058180 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8222557fef247e7803f51ab81619863)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:18.060249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:18.179822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8222557fef247e7803f51ab81619863', 'request_id': 'req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.186161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8222557fef247e7803f51ab81619863', 'request_id': 'req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.256056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8222557fef247e7803f51ab81619863', 'request_id': 'req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.268309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] The token cache is being invalidate because project f8222557fef247e7803f51ab81619863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:18.268610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.269269 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.270963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1317/2598] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:18 2022] DELETE /identity/v3/projects/f8222557fef247e7803f51ab81619863 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:18.275899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:18.291860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d61a1-f941-455e-bb95-35297a7947fe service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:18.358979 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:18.359873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:18.360161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:18.360442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] PATH_INFO: `/v3/users/fb0df214d711460a82fe2e9d66131894` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:18.361679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0df214d711460a82fe2e9d66131894)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:18.363712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:18.615928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0df214d711460a82fe2e9d66131894'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.629883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0df214d711460a82fe2e9d66131894'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.754885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0df214d711460a82fe2e9d66131894'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.755612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0df214d711460a82fe2e9d66131894'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.756310 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-b5d8b999-c679-423d-83c2-b535f9a411d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0df214d711460a82fe2e9d66131894'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:18.771445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1282/2599] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:18 2022] DELETE /identity/v3/users/fb0df214d711460a82fe2e9d66131894 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:19.308761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:19.310407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:19.326637 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a499ec9-1f0d-43dd-a969-bd2ba68121ad admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:19.392055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:19.393054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:19.393322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:19.393658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] PATH_INFO: `/v3/projects/0da774e4fea643e2aa0535e59214928d` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:19.394929 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da774e4fea643e2aa0535e59214928d)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:19.397045 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:19.581645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da774e4fea643e2aa0535e59214928d', 'request_id': 'req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:19.589746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da774e4fea643e2aa0535e59214928d', 'request_id': 'req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:19.591649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:19.675216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da774e4fea643e2aa0535e59214928d', 'request_id': 'req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:19.693492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] The token cache is being invalidate because project 0da774e4fea643e2aa0535e59214928d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:19.693492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:19.695874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-9fe0a331-7476-4e8e-b4ab-b89fcd98c233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:19.695874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1318/2600] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:19 2022] DELETE /identity/v3/projects/0da774e4fea643e2aa0535e59214928d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:19.701720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:19.702278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:19.702392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:19.721279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:19.722558 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:19.755791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:19.789919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1283/2601] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:19.797697 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:19.797992 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:19.798277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:19.820436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:19.826226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:19.830704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1319/2602] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:19.835828 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:19.846222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b61ca1-80d7-4f86-bedf-e3aab3afbd2d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:19.853025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:19.853769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:19.854060 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:19.854345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:19.855084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:19.855396 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:19.857137 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:19.870963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1284/2603] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:19.875753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:19.885549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950c7eb-d208-4d2d-94b0-6028183fbf43 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:19.890089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:19.890734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:19.891015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:19.891288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:19.891978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:19.893948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.037164 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1320/2604] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:19 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:20.041996 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:20.053803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331794b-d23e-4b5e-9f8f-889026b5925d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:20.118141 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:20.118896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.119194 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.119872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.120839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:20.124012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.230696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1285/2605] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:20 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:20.235714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:20.249368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9a87-486d-4f07-9ce7-1933b134c6e7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:20.253991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:20.254665 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.254951 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.255232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.255968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:20.257615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.271773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1321/2606] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:20.273683 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:20.284395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308089e-d99b-4a8e-9734-057273eb2819 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:20.290679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:20.291461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.291752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.292033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.292768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:20.294660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346f888-34c2-475f-9711-5dd4195ca042 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.314181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1286/2607] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:20.314461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:20.321613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0c979-11ad-4901-86af-e9c4aab51ceb admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:20.326187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:20.327095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.327403 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.327672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] PATH_INFO: `/v3/projects/353c5ba142db4b528a471ce75a9bfe9c/users/cb3b3bb510b341ad863e48b80748ac6e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.344946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=353c5ba142db4b528a471ce75a9bfe9c, user_id=cb3b3bb510b341ad863e48b80748ac6e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:20.347737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.464961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1322/2608] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:20 2022] PUT /identity/v3/projects/353c5ba142db4b528a471ce75a9bfe9c/users/cb3b3bb510b341ad863e48b80748ac6e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:20.475435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.475876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.476254 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.525873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] MFA Rules not processed for user `cb3b3bb510b341ad863e48b80748ac6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:20.527098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:20.614062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1287/2609] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:20.627313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:20.629997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:20.697696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662746b-b443-4331-8bff-ad6f317507ed admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:20.701493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:20.702197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:20.702491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:20.702775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:20.714253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:20.715720 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:20.725241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1323/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:23.428365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:23.429752 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:23.461411 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:23.487285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e1c3b-6d24-45da-9f18-3584d258ba6a None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:23.491055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:23.491908 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:23.492190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:23.492464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:23.494283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:23.566750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:23.568392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:23.585705 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1288/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:27.884606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:27.886378 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:27.902635 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc885f8-c884-48b5-a03d-b252b655943b service glance] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:27.907189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:27.908094 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:27.908405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:27.908682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] PATH_INFO: `/v3/users/391d41dab9294cd39d3b80d60570034e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:27.909931 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=391d41dab9294cd39d3b80d60570034e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:27.911855 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:28.111377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391d41dab9294cd39d3b80d60570034e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.112037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391d41dab9294cd39d3b80d60570034e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.206209 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391d41dab9294cd39d3b80d60570034e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.206849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391d41dab9294cd39d3b80d60570034e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.220670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-caa10404-e998-4def-bd8e-3c3b1b0454dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391d41dab9294cd39d3b80d60570034e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.229958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:INSERT=4,keystone:DELETE=11,keystone:SELECT=85 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:28.235076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1324/2612] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:27 2022] DELETE /identity/v3/users/391d41dab9294cd39d3b80d60570034e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:28.482608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:28.493755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245ebf0-c002-4643-b552-65cc7085d368 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:28.558056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:28.558927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:28.559217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:28.559499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] PATH_INFO: `/v3/projects/26ab6251a87c4271b77242545ef4837f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:28.560743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ab6251a87c4271b77242545ef4837f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:28.562780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:28.710524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ab6251a87c4271b77242545ef4837f', 'request_id': 'req-d670ed89-ac40-4af5-a96f-95a6c4821f72'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.810585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ab6251a87c4271b77242545ef4837f', 'request_id': 'req-d670ed89-ac40-4af5-a96f-95a6c4821f72'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.817225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ab6251a87c4271b77242545ef4837f', 'request_id': 'req-d670ed89-ac40-4af5-a96f-95a6c4821f72'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.834544 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] The token cache is being invalidate because project 26ab6251a87c4271b77242545ef4837f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:28.834962 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.835730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-d670ed89-ac40-4af5-a96f-95a6c4821f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:28.837897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1289/2613] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:28 2022] DELETE /identity/v3/projects/26ab6251a87c4271b77242545ef4837f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:28.843562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:28.844024 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:28.844313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:28.865418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:28.867408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:28.931843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1325/2614] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:28.939675 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:28.940288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:28.940634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:28.954446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:28.961751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:28.962972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:28.971354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1290/2615] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:28.975670 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:28.985949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbedb6b-42f0-4335-8117-2afc556d4f30 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:28.991589 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:28.992243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:28.992640 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:28.992925 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:28.993755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:28.994063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:28.995732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.011401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1326/2616] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:29.012815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.029425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473aedb-7a1e-4504-a15d-e2a5bc5336c1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.039437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.042823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.042823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.042823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.042823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.043499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.162462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1291/2617] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:29 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:29.166819 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.177196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d07940-ad8b-4d1e-89b5-db7ac3d34a8f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.235749 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.236336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.236619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.236894 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.237685 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.239727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.338440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1327/2618] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:29 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:29.342256 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.353496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795e91-151d-4db2-8dee-c34e7a515a4d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.358218 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.363665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.363665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.363665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.363665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.363665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.372893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1292/2619] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:29.377130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.387408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a9cc-0f01-441a-90c4-011eafa8d701 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.392442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.393109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.393450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.393755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.394482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.396087 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d33b1-0746-4db2-b846-485feb43ea28 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.406891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1328/2620] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:29.412907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.423262 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ead0a3-3654-46ab-97bd-44d6e3dfdc6c admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.427925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.429622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.429907 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.430183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] PATH_INFO: `/v3/projects/fa6fd86a75eb46ebb71de444cd5dd380/users/8b26bca4544b4931a1e032ebf0d3c3e3/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.448813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6fd86a75eb46ebb71de444cd5dd380, user_id=8b26bca4544b4931a1e032ebf0d3c3e3, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.452471 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.572776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1293/2621] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:29 2022] PUT /identity/v3/projects/fa6fd86a75eb46ebb71de444cd5dd380/users/8b26bca4544b4931a1e032ebf0d3c3e3/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:29.583563 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.583860 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.584145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.614491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] MFA Rules not processed for user `8b26bca4544b4931a1e032ebf0d3c3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:29.617409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:29.691811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:29.705932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1329/2622] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:29.708445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.776436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2356094-7631-47fd-9688-0a265ce5c952 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.781197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.781900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.782188 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.782681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.783286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.785217 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:29.883532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1294/2623] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:29 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:29.887266 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:29.898491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac86635-cfd2-4b56-92a5-d4b98795dfcf None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:29.955370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:29.956286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:29.956450 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:29.956678 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:29.957467 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:29.959495 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.071178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1330/2624] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:29 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:30.075239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:30.086768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88ab56-b407-418a-a520-5443ccb0b51b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:30.091682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:30.092339 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.092628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.092910 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.093670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:30.095330 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.107884 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1295/2625] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:30.112227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:30.123877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c86f02-211f-4cb6-a791-134e01943cd7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:30.128570 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:30.129478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.129778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.130058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] PATH_INFO: `/v3/projects/4bb03973435a4da5af10c8c47a302cb9/users/d857c305781e490894cec35a5bb3fac9/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.146285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb03973435a4da5af10c8c47a302cb9, user_id=d857c305781e490894cec35a5bb3fac9, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:30.149303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.261950 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1331/2626] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:30 2022] PUT /identity/v3/projects/4bb03973435a4da5af10c8c47a302cb9/users/d857c305781e490894cec35a5bb3fac9/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:30.266046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:30.276462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce0f8-43c1-4ce2-8102-9a5af0ba196b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:30.336880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:30.337480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.337765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.338042 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.338761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:30.340630 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598d424-2d79-48e2-b9b7-0bd2c16b6cd7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.351295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1296/2627] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:30.355236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:30.366303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baa938-1054-4dd9-b32d-f755c6e75e93 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:30.370866 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:30.371743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.372032 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.372311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] PATH_INFO: `/v3/projects/4bb03973435a4da5af10c8c47a302cb9/users/d857c305781e490894cec35a5bb3fac9/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.374603 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bb03973435a4da5af10c8c47a302cb9, user_id=d857c305781e490894cec35a5bb3fac9, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:30.377249 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.387865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1332/2628] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:30 2022] PUT /identity/v3/projects/4bb03973435a4da5af10c8c47a302cb9/users/d857c305781e490894cec35a5bb3fac9/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:30.402518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.402812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.403123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.428001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:30.432182 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] MFA Rules not processed for user `d857c305781e490894cec35a5bb3fac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:30.433474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:30.508575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1297/2629] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:30.521359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:30.528893 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:30.586235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b996d6-6cae-404f-bbfc-1048a63d5f4f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:30.590547 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:30.591123 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:30.591407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:30.591714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:30.602424 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:30.604252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:30.614145 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1333/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:31.964913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:31.974117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17671c-af41-4db3-9697-2b81254fffb0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:31.979506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd857c305781e490894cec35a5bb3fac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb03973435a4da5af10c8c47a302cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:31.980162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:31.980448 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:31.980731 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:31.983072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:31.983072 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:31.983537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:31.998928 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1298/2631] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:32.000021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:32.009645 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046e2d9-a91f-48b7-9dfb-722c7253f11d service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:32.020740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd857c305781e490894cec35a5bb3fac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb03973435a4da5af10c8c47a302cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:32.021462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:32.022309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:32.022896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:32.023347 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:32.025282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:32.120047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1334/2632] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:31 2022] POST /identity/v3/projects => generated 353 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:32.121588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:32.133628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa2014-23fc-44ed-ae4f-d402d92132bf tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:32.195694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd857c305781e490894cec35a5bb3fac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb03973435a4da5af10c8c47a302cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:32.196488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:32.196777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:32.197150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:32.197925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:32.198243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:32.199886 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:32.212257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1299/2633] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:36:32.254868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:32.265913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b331e-91e0-42f7-a397-520780a7f556 tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:32.270774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd857c305781e490894cec35a5bb3fac9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4bb03973435a4da5af10c8c47a302cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:32.271662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:32.272754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:32.273387 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] PATH_INFO: `/v3/projects/647af2430baa4ead8d8cfad66fb307e1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:32.274452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorizing `identity:delete_project(project_id=647af2430baa4ead8d8cfad66fb307e1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:32.276255 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:32.304710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:32.433654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647af2430baa4ead8d8cfad66fb307e1', 'request_id': 'req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:32.439156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647af2430baa4ead8d8cfad66fb307e1', 'request_id': 'req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:32.538504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647af2430baa4ead8d8cfad66fb307e1', 'request_id': 'req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:32.554496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] The token cache is being invalidate because project 647af2430baa4ead8d8cfad66fb307e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:32.554859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:32.555557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-b9d03303-aef2-4f9e-a4bd-b2e0584a8f4d tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:32.557769 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1335/2634] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:32 2022] DELETE /identity/v3/projects/647af2430baa4ead8d8cfad66fb307e1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:33.283183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:33.284936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:33.354130 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb7e4-b457-4451-8ae5-4667b9cd703b tempest-VolumeQuotaClassesTest-814197911 tempest-VolumeQuotaClassesTest-814197911-project-admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:33.386089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=100 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:33.418625 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:33.419502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:33.419794 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:33.420075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] PATH_INFO: `/v3/users/8b26bca4544b4931a1e032ebf0d3c3e3` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:33.421255 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b26bca4544b4931a1e032ebf0d3c3e3)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:33.423585 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:33.638930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b26bca4544b4931a1e032ebf0d3c3e3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:33.653049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b26bca4544b4931a1e032ebf0d3c3e3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:33.720327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b26bca4544b4931a1e032ebf0d3c3e3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:33.720976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b26bca4544b4931a1e032ebf0d3c3e3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:33.721642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b26bca4544b4931a1e032ebf0d3c3e3'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:33.735664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1300/2635] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:33 2022] DELETE /identity/v3/users/8b26bca4544b4931a1e032ebf0d3c3e3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:33.740267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:33.740910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:33.740910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:33.758317 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:33.759518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:33.823548 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1336/2636] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:33.833168 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:33.835102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:33.893332 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb462-429e-4011-9aae-76e63b5e6046 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:33.897565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:33.898197 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:33.901943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:33.901943 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:33.908019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:33.909559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:33.919354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1301/2637] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:34.449440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:34.457244 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:34.474585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ce3e7-93f8-4eca-944a-abb554ac3293 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:34.480194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:34.481391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:34.481682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:34.481985 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] PATH_INFO: `/v3/projects/fa6fd86a75eb46ebb71de444cd5dd380` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:34.483182 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6fd86a75eb46ebb71de444cd5dd380)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:34.485175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:34.617412 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6fd86a75eb46ebb71de444cd5dd380', 'request_id': 'req-7014a28f-3217-4410-9049-df5aaa6f6c4c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:34.623295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6fd86a75eb46ebb71de444cd5dd380', 'request_id': 'req-7014a28f-3217-4410-9049-df5aaa6f6c4c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:34.943811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6fd86a75eb46ebb71de444cd5dd380', 'request_id': 'req-7014a28f-3217-4410-9049-df5aaa6f6c4c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:34.961874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] The token cache is being invalidate because project fa6fd86a75eb46ebb71de444cd5dd380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:34.961874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:34.961874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:34.961874 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1337/2638] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:34 2022] DELETE /identity/v3/projects/fa6fd86a75eb46ebb71de444cd5dd380 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:34.963315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:34.972984 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda6c01-9390-483b-be10-3a0503160153 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:35.031753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:35.031753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:35.031966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:35.032222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:35.032501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] PATH_INFO: `/v3/users/d857c305781e490894cec35a5bb3fac9` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:35.034073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d857c305781e490894cec35a5bb3fac9)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:35.035997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:35.340693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd857c305781e490894cec35a5bb3fac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:35.352743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd857c305781e490894cec35a5bb3fac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:35.478615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd857c305781e490894cec35a5bb3fac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:35.479231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd857c305781e490894cec35a5bb3fac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:35.479876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9d5f6399-90aa-41f0-b0af-352511286902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd857c305781e490894cec35a5bb3fac9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:35.492693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1302/2639] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:34 2022] DELETE /identity/v3/users/d857c305781e490894cec35a5bb3fac9 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:35.944764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:35.946359 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:35.961300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a28f-3217-4410-9049-df5aaa6f6c4c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:35.997779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:36.026956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.027859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.028141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.028418 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] PATH_INFO: `/v3/projects/4bb03973435a4da5af10c8c47a302cb9` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.029691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bb03973435a4da5af10c8c47a302cb9)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.032919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.190990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bb03973435a4da5af10c8c47a302cb9', 'request_id': 'req-eb42880c-10cb-4933-8190-9cd09e2755ac'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:36.197634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bb03973435a4da5af10c8c47a302cb9', 'request_id': 'req-eb42880c-10cb-4933-8190-9cd09e2755ac'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:36.310671 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bb03973435a4da5af10c8c47a302cb9', 'request_id': 'req-eb42880c-10cb-4933-8190-9cd09e2755ac'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:36.315773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] The token cache is being invalidate because project 4bb03973435a4da5af10c8c47a302cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:36.316063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:36.316727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-eb42880c-10cb-4933-8190-9cd09e2755ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:36.318277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1338/2640] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:35 2022] DELETE /identity/v3/projects/4bb03973435a4da5af10c8c47a302cb9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:36.323977 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.324492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.324617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.343965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:36.344880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:36.411897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1303/2641] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:36.421217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.421533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.421897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.442054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:36.443277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:36.452610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1339/2642] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:36.456976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.467063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554e8dc-f165-46bd-a389-420ba3999692 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.471740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.472393 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.472727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.473015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.473790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:36.474105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.475729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.487971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1304/2643] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:36.493035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.503432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10b0c1-78d9-4574-8b4b-cf0cc9cba77a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.509724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.510429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.510729 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.511015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.511743 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.514881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.621926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1340/2644] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:36 2022] POST /identity/v3/projects => generated 407 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:36.626338 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.637671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536d90c-a64c-4630-8310-415c049fe29b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.666407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.668125 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:36.692370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:36.699546 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.700223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.700507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.700791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.701515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.707120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.734457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1abaf-6cbd-41e0-90ab-9f27f56fcff6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.741703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.741703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.741703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.741703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] PATH_INFO: `/v3/users/13b1d6eadd92489ca35592665fbe71ca` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.742943 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b1d6eadd92489ca35592665fbe71ca)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.744981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.801877 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1305/2645] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:36 2022] POST /identity/v3/users => generated 304 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:36.803280 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.816297 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b574f95-b996-4c73-94f7-4a7b523be804 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.823423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.824109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.824398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.824678 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.825740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.827391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.841114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1306/2646] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:36.845409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.857105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aae6a9-95c7-4f76-92ae-140fd1507cb0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.861939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.862596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.862927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.863189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.863914 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.865537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:36.877225 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1307/2647] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:36.881424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:36.891236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a732-0f14-4669-bae5-1743f2d8c8b0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:36.896017 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:36.896897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:36.897183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:36.897507 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] PATH_INFO: `/v3/projects/1ad190546e8d4b2081db9d44897643ed/users/9010d46d7a7b407e97c8e32c810a9a4e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:36.912131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ad190546e8d4b2081db9d44897643ed, user_id=9010d46d7a7b407e97c8e32c810a9a4e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:36.915492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c36df0-930a-45c0-9be7-4a6f848e2c40 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.041063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:37.049282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b1d6eadd92489ca35592665fbe71ca'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:37.049960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b1d6eadd92489ca35592665fbe71ca'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:37.075769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1308/2648] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:36 2022] PUT /identity/v3/projects/1ad190546e8d4b2081db9d44897643ed/users/9010d46d7a7b407e97c8e32c810a9a4e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:37.086210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.086210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.086502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.113237 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] MFA Rules not processed for user `9010d46d7a7b407e97c8e32c810a9a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:37.114517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:37.187922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1309/2649] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:37.192681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.194165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:37.212795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b1d6eadd92489ca35592665fbe71ca'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:37.213492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b1d6eadd92489ca35592665fbe71ca'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:37.227330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-8f049317-57f2-406a-b3f7-7583f6dc7abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b1d6eadd92489ca35592665fbe71ca'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:37.245903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1341/2650] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:36 2022] DELETE /identity/v3/users/13b1d6eadd92489ca35592665fbe71ca => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:37.250739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.251035 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.251326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.264632 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a2513-3bbc-4d42-96d6-fc7e11955fa1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.272910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:37.272910 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:37.298772 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:37.323041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.323729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.324022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.324580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.325002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.326991 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.332737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1342/2651] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:37.343027 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.344392 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:37.400984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab5752-a0df-4a52-a49b-5ca466afe910 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.404759 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.405446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.405738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.406022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.416915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.418394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.428119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1343/2652] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:37.439934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1310/2653] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:37 2022] POST /identity/v3/projects => generated 409 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:36:37.444002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.453774 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd37-43d9-4eb8-b4a0-ea27193b8384 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.508540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:37.514746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.516534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.516534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.516534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.516892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.518732 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.603241 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1344/2654] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:37 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:37.604455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.615501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae4d44-1be1-48c8-9cad-5a7a4bc700a5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.620973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.621881 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.622169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.622446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.623172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.625073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.637333 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1311/2655] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:37.643330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.654539 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0df422-56ff-4d3e-9264-43266859b55b admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.660042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.660930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.661216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.661545 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] PATH_INFO: `/v3/projects/10ad5760ed2a4806828109c554239eab/users/276d997a5bb849c0a5ce359df929d3b6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.677140 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ad5760ed2a4806828109c554239eab, user_id=276d997a5bb849c0a5ce359df929d3b6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.680198 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.790509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1345/2656] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:37 2022] PUT /identity/v3/projects/10ad5760ed2a4806828109c554239eab/users/276d997a5bb849c0a5ce359df929d3b6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:37.794624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.805421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69355-5317-4905-8999-463b128f2fac admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.851843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.851843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:37.869531 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.870018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.870492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.874960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.874960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.874960 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.887857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1312/2657] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:36:37.892435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:37.906150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc1525-3d32-41e7-a752-e6a818c9f672 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.913155 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:37.914561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.914854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.915131 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] PATH_INFO: `/v3/projects/10ad5760ed2a4806828109c554239eab/users/276d997a5bb849c0a5ce359df929d3b6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:37.924276 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ad5760ed2a4806828109c554239eab, user_id=276d997a5bb849c0a5ce359df929d3b6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:37.927324 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7f0f0-2a10-431b-b9a1-0241f1fa61f5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:37.945199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1313/2658] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:37 2022] PUT /identity/v3/projects/10ad5760ed2a4806828109c554239eab/users/276d997a5bb849c0a5ce359df929d3b6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:37.950316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780148e-e3c3-42b2-b9f8-e2ea4c532230 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:37.970774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:37.971069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:37.971499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:38.000040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] MFA Rules not processed for user `276d997a5bb849c0a5ce359df929d3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:38.001694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:38.018478 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:38.024080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:38.024080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:38.024080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] PATH_INFO: `/v3/projects/4189389e4d434140ac8b6a05de9f4bcf` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:38.024080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4189389e4d434140ac8b6a05de9f4bcf)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:38.024473 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:38.072732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1314/2659] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:38.084127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:38.086111 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:38.107354 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:38.143899 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4189389e4d434140ac8b6a05de9f4bcf', 'request_id': 'req-4efcaaff-b730-44ae-8253-8882c9894c46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.150500 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4189389e4d434140ac8b6a05de9f4bcf', 'request_id': 'req-4efcaaff-b730-44ae-8253-8882c9894c46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.150994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d67661-90cb-4a27-9eab-6a6f6314f3fd None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:38.199192 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:38.199823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:38.200106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:38.200384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:38.264579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:38.266421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:38.278627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1315/2660] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:38.426452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4189389e4d434140ac8b6a05de9f4bcf', 'request_id': 'req-4efcaaff-b730-44ae-8253-8882c9894c46'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.426452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:38.426452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] The token cache is being invalidate because project 4189389e4d434140ac8b6a05de9f4bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:38.426452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.426452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.427058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1346/2661] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:37 2022] DELETE /identity/v3/projects/4189389e4d434140ac8b6a05de9f4bcf => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:38.430509 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:38.491556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac810a8-4e3a-4f02-a0af-8d51db7a40e4 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:38.497056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:38.498027 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:38.498315 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:38.498849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] PATH_INFO: `/v3/users/fe2d6a0022574e928061503d7bbcae64` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:38.500039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe2d6a0022574e928061503d7bbcae64)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:38.502026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:38.635732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:38.741501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe2d6a0022574e928061503d7bbcae64'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.754257 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe2d6a0022574e928061503d7bbcae64'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.804809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:38.806235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:38.818303 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcaaff-b730-44ae-8253-8882c9894c46 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:38.822190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:38.822828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:38.823108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:38.823384 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:38.825092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:38.834668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe2d6a0022574e928061503d7bbcae64'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.835383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe2d6a0022574e928061503d7bbcae64'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.836976 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe2d6a0022574e928061503d7bbcae64'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:38.850203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1316/2662] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:38 2022] DELETE /identity/v3/users/fe2d6a0022574e928061503d7bbcae64 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:38.886072 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:38.887537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cf92-c49f-4d55-adf2-1df56bd282ca service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:38.897593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1347/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:36:39.369432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:39.371575 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:39.443698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0afb0-c073-47a4-87cc-c68cb951c5f1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:39.448436 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:39.449296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:39.449618 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:39.449924 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] PATH_INFO: `/v3/projects/0255a41891e746bf9236a8267de8609f` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:39.451167 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0255a41891e746bf9236a8267de8609f)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:39.453093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:39.647114 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0255a41891e746bf9236a8267de8609f', 'request_id': 'req-6e8ae0c6-b225-4cba-a589-aaf070187f95'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:39.793539 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0255a41891e746bf9236a8267de8609f', 'request_id': 'req-6e8ae0c6-b225-4cba-a589-aaf070187f95'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:39.800926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0255a41891e746bf9236a8267de8609f', 'request_id': 'req-6e8ae0c6-b225-4cba-a589-aaf070187f95'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:39.820542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] The token cache is being invalidate because project 0255a41891e746bf9236a8267de8609f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:39.821387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:39.822317 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-6e8ae0c6-b225-4cba-a589-aaf070187f95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:39.825144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1317/2664] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:39 2022] DELETE /identity/v3/projects/0255a41891e746bf9236a8267de8609f => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:39.831297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:39.831809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:39.832261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:39.856778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:39.858075 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:39.930737 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1348/2665] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:39.938424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:39.938722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:39.939123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:39.976494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:39.978906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:39.992815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1318/2666] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:39.997114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.010050 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28d9e3-b59f-468f-996c-0b08bd181625 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.015989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.016781 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.017442 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.017733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.018485 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:40.018883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.020428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.041569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1349/2667] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:40.042634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.062433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003a5cf-258f-4c07-ac2f-92de6a5751dc None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.067906 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.068677 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.068990 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.069389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.080261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.080261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.232073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1319/2668] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:36:40.236068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.247383 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999b6bd-0e75-4f8d-aa19-3ed4f0b952a6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.309948 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.310639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.311132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.311429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.312141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.314175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.406858 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1350/2669] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:40 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:40.411118 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.418053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:40.421868 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60cc02-e780-4273-b717-eadac67327d5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.426551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.427208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.427498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.427776 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.428568 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.430682 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.441729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1320/2670] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:40.449569 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.460115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8622e-4c6d-43cc-80da-42c6ce3808a9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.465590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.466286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.466574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.469310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.469310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.469310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ebb85-cf3e-4f26-b098-8a38be223ee7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.483856 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1351/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:40.485019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.496956 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c572c90-4d04-471b-a5a1-6e4c59118947 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.502183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.503292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.503593 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.503857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] PATH_INFO: `/v3/projects/43338785ced645639bb4f339f9b52cad/users/71c826d0cd104ecb8a9c8fbd2741aeb2/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.520501 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43338785ced645639bb4f339f9b52cad, user_id=71c826d0cd104ecb8a9c8fbd2741aeb2, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.523846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.625829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1321/2672] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:40 2022] PUT /identity/v3/projects/43338785ced645639bb4f339f9b52cad/users/71c826d0cd104ecb8a9c8fbd2741aeb2/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:40.637397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.637778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.638130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.645400 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:40.666402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] MFA Rules not processed for user `71c826d0cd104ecb8a9c8fbd2741aeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:40.667659 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:40.731817 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1352/2673] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:40.736485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.802062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda3c5f-482d-464c-bb15-de8cb3b0bca5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:40.807391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:40.808222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:40.808511 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:40.808792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:40.809525 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:40.811493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:40.949160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1322/2674] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:40.954491 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:40.967441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fd193-277d-40dc-be3e-1c3510a34898 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.026191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.026939 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.027237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.027525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.028225 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.030228 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.181847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1353/2675] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:40 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:41.186139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:41.197981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee5d9b-b3a4-4334-82ab-0f43f3af0ff6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.202753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.203454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.203730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.203998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.204722 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.206373 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.218189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1323/2676] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:41.222977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:41.234892 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dda65-19a4-4f0f-9ba0-8f1beef2c86a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.239797 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.240674 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.240956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.241231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] PATH_INFO: `/v3/projects/7f51707e654d41b3aadc10635c4685b9/users/42db7dff61e3438db72c33b3c105dd48/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.265296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f51707e654d41b3aadc10635c4685b9, user_id=42db7dff61e3438db72c33b3c105dd48, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.268099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.384437 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1354/2677] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:41 2022] PUT /identity/v3/projects/7f51707e654d41b3aadc10635c4685b9/users/42db7dff61e3438db72c33b3c105dd48/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:41.388589 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:41.398846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c40aa-68ea-4ec5-a4b9-dbe121966580 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.456918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.457723 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.458014 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.458296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.459011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.460654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9164a8a-d909-483f-94c2-1ca6266bda8f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.470679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1324/2678] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:41.474822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:41.490240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bee90a-1528-4244-9901-1057e5ad5de7 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.492976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.494067 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.494333 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.494672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] PATH_INFO: `/v3/projects/7f51707e654d41b3aadc10635c4685b9/users/42db7dff61e3438db72c33b3c105dd48/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.496691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f51707e654d41b3aadc10635c4685b9, user_id=42db7dff61e3438db72c33b3c105dd48, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.499619 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.511748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1355/2679] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:41 2022] PUT /identity/v3/projects/7f51707e654d41b3aadc10635c4685b9/users/42db7dff61e3438db72c33b3c105dd48/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:41.534769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.535261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.535383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.563319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] MFA Rules not processed for user `42db7dff61e3438db72c33b3c105dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:41.564532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:41.594647 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:41.635995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1325/2680] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:41.647336 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:41.649712 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:41.705657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:41.712189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9984a2-b74c-425d-93bb-1404f1c18af6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:41.717079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:41.717810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:41.718112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:41.718409 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:41.774506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:41.775964 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924e926-54f3-4445-8306-85a28a2efb61 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:41.785124 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1356/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:46.298051 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:46.299902 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:46.387493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac086-5da4-41c3-989f-b9a4469b8cf9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:46.393553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:46.394395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:46.394685 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:46.394946 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] PATH_INFO: `/v3/users/cb3b3bb510b341ad863e48b80748ac6e` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:46.396166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3b3bb510b341ad863e48b80748ac6e)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:46.398769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:46.641358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3b3bb510b341ad863e48b80748ac6e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:46.655377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3b3bb510b341ad863e48b80748ac6e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:46.750913 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3b3bb510b341ad863e48b80748ac6e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:46.751258 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3b3bb510b341ad863e48b80748ac6e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:46.752099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3b3bb510b341ad863e48b80748ac6e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:46.766244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1326/2682] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:46 2022] DELETE /identity/v3/users/cb3b3bb510b341ad863e48b80748ac6e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:46.771197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:46.771981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:46.772161 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:46.790877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:46.792095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:46.856188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1357/2683] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:46.867336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:46.868771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:46.916202 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:46.927423 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1f78-d22f-4820-bc24-6d1b36b0c9c6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:46.931113 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:46.931764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:46.932055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:46.940783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:46.941833 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:46.943290 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5da4c-8635-4281-aa2b-31623eb58daf service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:46.953099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1327/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:47.425461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:47.426982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:47.440099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401d74-1e1f-44e3-8f8f-06a108ee57b8 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:47.445469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:47.445888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:47.446232 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:47.446515 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] PATH_INFO: `/v3/projects/353c5ba142db4b528a471ce75a9bfe9c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:47.447734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=353c5ba142db4b528a471ce75a9bfe9c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:47.449708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:47.561427 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353c5ba142db4b528a471ce75a9bfe9c', 'request_id': 'req-648227f4-2b1e-4e3b-b359-12b520b17d2b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:47.567115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353c5ba142db4b528a471ce75a9bfe9c', 'request_id': 'req-648227f4-2b1e-4e3b-b359-12b520b17d2b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:47.640278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353c5ba142db4b528a471ce75a9bfe9c', 'request_id': 'req-648227f4-2b1e-4e3b-b359-12b520b17d2b'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:47.651991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] The token cache is being invalidate because project 353c5ba142db4b528a471ce75a9bfe9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:47.652286 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:47.652915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-648227f4-2b1e-4e3b-b359-12b520b17d2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:47.655010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1358/2685] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:47 2022] DELETE /identity/v3/projects/353c5ba142db4b528a471ce75a9bfe9c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:47.659965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:47.660500 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:47.660599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:47.682941 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:47.684214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:47.751903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1328/2686] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:47.759647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:47.760060 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:47.760344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:47.779825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:47.781030 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:47.788836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1359/2687] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:47.793005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:47.803756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4503cf-c4ed-49c2-aadc-57da89e927c6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:47.808360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:47.809004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:47.809291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:47.809599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:47.811191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:47.811191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:47.812320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:47.823964 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1329/2688] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:36:47.828696 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:47.838482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41961f07-1b83-4803-b4a2-fe2fe1254136 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:47.844058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:47.844713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:47.844998 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:47.845276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:47.846155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:47.848070 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:47.862818 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:48.008033 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1360/2689] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:47 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:48.012239 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:48.022982 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b420bd3-3797-41ee-9185-eddcb41f57d0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:48.080387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:48.081139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.081450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.081734 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.082432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:48.084359 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:48.212560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1330/2690] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:48 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:48.217169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:48.227322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d2451-4f69-4ec3-b4eb-bcfa9b12a5be admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:48.231830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:48.234947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.234947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.234947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.234947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:48.235408 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:48.245802 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1361/2691] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:48.250097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:48.259919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbef3d1-bb4e-417a-b046-ed00830e7a16 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:48.265233 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:48.265918 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.266204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.266480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.267191 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:48.268768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ed06-d2df-4c39-99d2-3dff11ba8a8e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:48.279505 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1331/2692] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:48.283585 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:48.293606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0b556-c330-4d98-83ae-0b069d407858 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:48.298730 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:48.299609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.299896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.300173 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] PATH_INFO: `/v3/projects/3f619e30f070457f8f85738785898964/users/86c1b1fe81704163895633977107c301/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.316475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f619e30f070457f8f85738785898964, user_id=86c1b1fe81704163895633977107c301, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:48.319150 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:48.424574 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1362/2693] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:48 2022] PUT /identity/v3/projects/3f619e30f070457f8f85738785898964/users/86c1b1fe81704163895633977107c301/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:48.436455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.436749 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.437037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.464552 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] MFA Rules not processed for user `86c1b1fe81704163895633977107c301`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:48.465736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:48.508764 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:48.534165 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1332/2694] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:48.545183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:48.550112 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:48.604577 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd09ae-04ad-4650-8b41-34daca104ed8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:48.608770 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:48.609469 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:48.609761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:48.610042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:48.618399 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:48.619842 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:48.631186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1363/2695] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:50.462032 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:50.463468 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:50.475949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343898c-caa9-4eed-88d3-9d68aa68e440 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:50.479547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:50.480223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:50.480538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:50.480911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:50.483099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:50.555164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:50.556655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7c990-10d5-40a8-8729-83d133d593db service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:50.567729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1333/2696] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:51.657533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:51.659001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:51.730476 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290688e-43fc-4e3e-a583-8c23e5d48a90 service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:51.735996 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:51.736872 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:51.737155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:51.737461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] PATH_INFO: `/v3/users/9010d46d7a7b407e97c8e32c810a9a4e` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:51.738646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9010d46d7a7b407e97c8e32c810a9a4e)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:51.740688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:51.945582 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9010d46d7a7b407e97c8e32c810a9a4e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:51.946280 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9010d46d7a7b407e97c8e32c810a9a4e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.017282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9010d46d7a7b407e97c8e32c810a9a4e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.017933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9010d46d7a7b407e97c8e32c810a9a4e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.026098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:52.030750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9010d46d7a7b407e97c8e32c810a9a4e'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.042767 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1364/2697] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:51 2022] DELETE /identity/v3/users/9010d46d7a7b407e97c8e32c810a9a4e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:52.048059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:52.048536 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:52.048636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:52.066897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:52.068219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:52.133798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1334/2698] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:52.143675 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:52.145099 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:52.204083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79813e-0b9a-4b0e-9dd6-60a9fa8ecfd0 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:52.207863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:52.208511 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:52.208883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:52.209151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:52.219734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:52.221183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:52.231055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1365/2699] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:52.636249 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:52.637795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:52.650791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bcc5c-b849-4ee7-b47d-fd0da5d22395 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:52.656942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:52.661311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:52.661311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:52.661311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] PATH_INFO: `/v3/projects/1ad190546e8d4b2081db9d44897643ed` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:52.662107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ad190546e8d4b2081db9d44897643ed)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:52.665055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:52.802517 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ad190546e8d4b2081db9d44897643ed', 'request_id': 'req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.869011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ad190546e8d4b2081db9d44897643ed', 'request_id': 'req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.876059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ad190546e8d4b2081db9d44897643ed', 'request_id': 'req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.884054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:52.887813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] The token cache is being invalidate because project 1ad190546e8d4b2081db9d44897643ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:52.888762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.889466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:52.891022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1335/2700] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:52 2022] DELETE /identity/v3/projects/1ad190546e8d4b2081db9d44897643ed => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:52.895052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:52.906090 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd98f-713f-4622-9b1d-763940499be6 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:52.970309 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:52.971379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:52.971672 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:52.971961 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] PATH_INFO: `/v3/users/276d997a5bb849c0a5ce359df929d3b6` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:52.973148 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=276d997a5bb849c0a5ce359df929d3b6)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:52.975130 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:53.208566 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276d997a5bb849c0a5ce359df929d3b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:53.209212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276d997a5bb849c0a5ce359df929d3b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:53.280981 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276d997a5bb849c0a5ce359df929d3b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:53.281688 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276d997a5bb849c0a5ce359df929d3b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:53.294926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4ce0c623-8a4d-421a-b90d-84b2166bccdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276d997a5bb849c0a5ce359df929d3b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:53.307178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1366/2701] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:52 2022] DELETE /identity/v3/users/276d997a5bb849c0a5ce359df929d3b6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:53.779518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:53.781216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:53.795275 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f4eea-4dc8-4687-aeb7-a3a5e3ea4cca admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:53.856126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:53.857636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:53.857929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:53.858215 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] PATH_INFO: `/v3/projects/10ad5760ed2a4806828109c554239eab` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:53.859428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ad5760ed2a4806828109c554239eab)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:53.861415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:53.989535 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ad5760ed2a4806828109c554239eab', 'request_id': 'req-361b2d49-3ef6-4bab-ac60-4eba58d616fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:54.076148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ad5760ed2a4806828109c554239eab', 'request_id': 'req-361b2d49-3ef6-4bab-ac60-4eba58d616fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:54.082576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ad5760ed2a4806828109c554239eab', 'request_id': 'req-361b2d49-3ef6-4bab-ac60-4eba58d616fd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:54.094728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] The token cache is being invalidate because project 10ad5760ed2a4806828109c554239eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:54.095022 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:54.095707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-361b2d49-3ef6-4bab-ac60-4eba58d616fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:54.097432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1336/2702] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:53 2022] DELETE /identity/v3/projects/10ad5760ed2a4806828109c554239eab => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:54.104057 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.104552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.104662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.112083 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:54.123632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:54.124830 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:54.185082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1367/2703] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:54.192966 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.193306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.193627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.212377 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:54.213639 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:54.222491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1337/2704] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:54.226877 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.237196 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fc679-4fed-4015-ada1-6b0fb3a2000c None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.242011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.242669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.242959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.243267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.243995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:36:54.244316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.246095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.258710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1368/2705] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:36:54.263561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.274007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2f479-ef40-46b7-82d7-c9eb713c2096 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.279298 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.283180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.283180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.283180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.283180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.283547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.397378 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1338/2706] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:54.401319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.411773 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837f20c-3719-4d2a-af37-a9f1f8d42060 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.469452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.470146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.470434 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.470715 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.471413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.473428 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.604999 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1369/2707] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:54.606374 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.617863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c165cf-5995-4400-931c-ea619200ccc3 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.623935 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.624657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.624951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.625236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.626055 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.628933 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.640484 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1339/2708] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:54.644787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.657199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4d91b-7af5-49f7-86bd-8306953ec2d6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.662328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.662982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.663297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.663551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.664311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.666042 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02b323-7cff-42d1-93aa-315b459da842 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.678463 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1370/2709] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:54.682688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.694322 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b1e15-79d5-4765-8c09-0a4255f466d0 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:54.699311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:54.700301 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.700599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.701037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] PATH_INFO: `/v3/projects/fa5ca372d10d49a09d692fcd8f341f55/users/d10229adb6e04ad0be397538b7b641c0/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.717247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5ca372d10d49a09d692fcd8f341f55, user_id=d10229adb6e04ad0be397538b7b641c0, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:54.720265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:54.810782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1340/2710] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:54 2022] PUT /identity/v3/projects/fa5ca372d10d49a09d692fcd8f341f55/users/d10229adb6e04ad0be397538b7b641c0/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:54.821487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:54.821786 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:54.822109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:54.849089 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] MFA Rules not processed for user `d10229adb6e04ad0be397538b7b641c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:54.850351 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:54.914245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:54.918259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1371/2711] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:54.925992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:54.962304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:54.994102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8575cb-5071-4d59-8fc9-998ff63d5174 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.000085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.000736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.001019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.001293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.002002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.003961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.132102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1341/2712] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:55.136365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.148862 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1ecd5-11ed-4484-a536-3843c0fc64b1 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.214995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.221549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.221549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.221549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.221549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.221549 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.314529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1372/2713] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:36:55 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:36:55.321718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.332690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffac4b-c048-4f1b-acd0-8796929f0660 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.337384 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.338058 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.338344 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.338623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.339611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.341466 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.353216 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1342/2714] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:55.358355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.368709 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc90ddc-6d96-46db-b836-aab8fb2bcf14 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.374537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.375765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.375765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.375997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] PATH_INFO: `/v3/projects/1681694ef8c947ed897773b1beee0eba/users/dfee2b2db4d4494abd135d90ca25bca6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.394308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1681694ef8c947ed897773b1beee0eba, user_id=dfee2b2db4d4494abd135d90ca25bca6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.397273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.534013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1373/2715] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:55 2022] PUT /identity/v3/projects/1681694ef8c947ed897773b1beee0eba/users/dfee2b2db4d4494abd135d90ca25bca6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:36:55.538251 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.548532 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246608c4-ebe1-4367-b4f5-90689e35832e admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.620336 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.621281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.621653 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.622128 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.623033 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.625319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8ff4e-08ea-4ac0-8d7c-563f16f8889b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.640282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1343/2716] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:36:55.642405 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.658504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2554f-444e-444f-8350-dab3fd294d15 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.665587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.666572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.666963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.667323 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] PATH_INFO: `/v3/projects/1681694ef8c947ed897773b1beee0eba/users/dfee2b2db4d4494abd135d90ca25bca6/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.670138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1681694ef8c947ed897773b1beee0eba, user_id=dfee2b2db4d4494abd135d90ca25bca6, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.672826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.685628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1374/2717] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:36:55 2022] PUT /identity/v3/projects/1681694ef8c947ed897773b1beee0eba/users/dfee2b2db4d4494abd135d90ca25bca6/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:55.696640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.696940 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.697651 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.726024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] MFA Rules not processed for user `dfee2b2db4d4494abd135d90ca25bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:55.727896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:55.810449 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1344/2718] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:36:55.822008 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:55.834390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:55.887542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452501a3-01a3-4126-9288-b5832f529f69 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:55.892583 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:55.893247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:55.893765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:55.894302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:55.909363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:55.910963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db08e1-19ee-4cb9-8f87-efc8a7989dc1 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:55.923104 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1375/2719] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:57.021899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:57.048236 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:57.084963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ba3a8-6ef6-4925-bc84-22effc814261 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:57.089921 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:57.090788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:57.092208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:57.092502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] PATH_INFO: `/v3/users/d10229adb6e04ad0be397538b7b641c0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:57.093724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10229adb6e04ad0be397538b7b641c0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:57.095664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:57.294591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10229adb6e04ad0be397538b7b641c0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:57.306987 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10229adb6e04ad0be397538b7b641c0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:57.386541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10229adb6e04ad0be397538b7b641c0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:57.387295 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10229adb6e04ad0be397538b7b641c0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:57.389112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10229adb6e04ad0be397538b7b641c0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:57.402622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1345/2720] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:57 2022] DELETE /identity/v3/users/d10229adb6e04ad0be397538b7b641c0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:57.407984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:57.408496 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:57.408611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:57.427329 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:36:57.428726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:57.480137 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:57.492396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1376/2721] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:36:57.507318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:57.508762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:57.572919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f6894-4d2b-48ca-8208-557a5d175c16 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:57.577176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:57.577863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:57.578143 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:57.578420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:57.587660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:57.589091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:57.599522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1346/2722] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:36:58.112185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:58.113810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:58.140695 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3f640-139c-4569-a3f9-9e309bced388 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:58.155471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:58.156586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:58.157025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:58.157477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] PATH_INFO: `/v3/projects/fa5ca372d10d49a09d692fcd8f341f55` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:58.159556 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5ca372d10d49a09d692fcd8f341f55)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:58.163029 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:58.564938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5ca372d10d49a09d692fcd8f341f55', 'request_id': 'req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:58.570474 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5ca372d10d49a09d692fcd8f341f55', 'request_id': 'req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:58.689989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5ca372d10d49a09d692fcd8f341f55', 'request_id': 'req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:58.702210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] The token cache is being invalidate because project fa5ca372d10d49a09d692fcd8f341f55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:36:58.706502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:58.706502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:58.706502 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1377/2723] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:58 2022] DELETE /identity/v3/projects/fa5ca372d10d49a09d692fcd8f341f55 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:58.709011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:58.720355 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43205-a4fe-401a-982b-7d08214f0407 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:58.765427 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:36:58.781478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:58.782435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:58.782719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:58.783005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] PATH_INFO: `/v3/users/dfee2b2db4d4494abd135d90ca25bca6` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:58.784499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfee2b2db4d4494abd135d90ca25bca6)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:58.786524 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:59.031565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfee2b2db4d4494abd135d90ca25bca6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.043268 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfee2b2db4d4494abd135d90ca25bca6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.143184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfee2b2db4d4494abd135d90ca25bca6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.143826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfee2b2db4d4494abd135d90ca25bca6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.144440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfee2b2db4d4494abd135d90ca25bca6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.157767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1347/2724] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:58 2022] DELETE /identity/v3/users/dfee2b2db4d4494abd135d90ca25bca6 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:36:59.563586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:59.565037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:59.577812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1efd9-4a4c-4904-bcd3-0bc5c10db144 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:59.608950 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:36:59.610738 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:36:59.643095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:59.645917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:59.645917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:59.645917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] PATH_INFO: `/v3/users/86c1b1fe81704163895633977107c301` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:59.645917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c1b1fe81704163895633977107c301)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:59.647978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:59.685101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a2153-5a7d-4edb-a8ee-f2652b7fca65 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:36:59.690110 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:36:59.691002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:36:59.691478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:36:59.691763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] PATH_INFO: `/v3/projects/1681694ef8c947ed897773b1beee0eba` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:36:59.692965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1681694ef8c947ed897773b1beee0eba)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:36:59.694925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:36:59.884621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c1b1fe81704163895633977107c301'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.885306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c1b1fe81704163895633977107c301'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.885740 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1681694ef8c947ed897773b1beee0eba', 'request_id': 'req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.981437 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1681694ef8c947ed897773b1beee0eba', 'request_id': 'req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.987923 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1681694ef8c947ed897773b1beee0eba', 'request_id': 'req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:36:59.990208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:00.000112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] The token cache is being invalidate because project 1681694ef8c947ed897773b1beee0eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:00.000504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:00.001162 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-90f2e8b3-4ce3-4191-862e-c6b3579fb25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:00.002725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1348/2725] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:36:59 2022] DELETE /identity/v3/projects/1681694ef8c947ed897773b1beee0eba => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:00.008767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31ed-b245-4a99-970c-c2a031b1b653 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.009052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31ed-b245-4a99-970c-c2a031b1b653 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.009371 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba31ed-b245-4a99-970c-c2a031b1b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.032073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0fba31ed-b245-4a99-970c-c2a031b1b653 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:00.033481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0fba31ed-b245-4a99-970c-c2a031b1b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.061401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c1b1fe81704163895633977107c301'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:00.062270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c1b1fe81704163895633977107c301'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:00.078472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-03623ce5-bff8-45cd-a682-9bfa0ca3f55e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c1b1fe81704163895633977107c301'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:00.087002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:00.093791 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1378/2726] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:36:59 2022] DELETE /identity/v3/users/86c1b1fe81704163895633977107c301 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:00.098523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.098879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.099216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.101617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1349/2727] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:00.113842 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.114139 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.114522 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.121521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:00.122956 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.139853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:00.141814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.204691 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1379/2728] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:00.214542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.216626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.221530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1350/2729] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:00.235122 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.247296 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2a725-8dfb-43a4-abfd-ca500c34069f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.252850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.253550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.253870 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.254158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.254890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:00.255203 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.256856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.272102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1351/2730] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:37:00.281105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef6e2d-b071-4e94-94b2-e57bc7584166 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.281719 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.284965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.285657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.285947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.286257 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.291774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5474d-710e-42a8-b0e0-4575757d8f24 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.296339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.296718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.297409 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.297728 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.297865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.298001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.298716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.300615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.308716 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1380/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:37:00.476751 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1352/2732] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:37:00.480779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.490954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a5d3-36f6-41f4-89d2-6c5c1dce05b8 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.562349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.562807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.563558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.563558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.565022 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.567505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.639714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1381/2733] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:00.640930 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.651504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f212483-13da-4449-9b0c-f6be3e5d76a1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.656464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.657204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.657527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.657809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.658538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.660150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.674792 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1353/2734] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:00.680361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.688787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52cac-d0a7-4ed8-ac13-8bb1f8534a65 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.693965 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.694733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.695025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.695305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.696382 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.698101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.710753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1382/2735] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:00.715145 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.725876 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5431-fe0c-4375-9e14-976d2a93249d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.730687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.732057 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.737375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.737375 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] PATH_INFO: `/v3/projects/fc7c8936576b4df38c281c1aa4df5e2d/users/00105db60ffc41bcbd3d70ec9b4429c1/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.748002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7c8936576b4df38c281c1aa4df5e2d, user_id=00105db60ffc41bcbd3d70ec9b4429c1, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.751121 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2274d0c-65fe-43f6-a7eb-9ed775904c92 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.823784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.824870 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.840108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307bde-8ef1-412d-9d0e-1e30626a8da1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:00.845194 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:00.846710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.846710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.847220 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] PATH_INFO: `/v3/projects/3f619e30f070457f8f85738785898964` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.851444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f619e30f070457f8f85738785898964)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:00.851444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:00.874521 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:00.879732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1354/2736] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:00 2022] PUT /identity/v3/projects/fc7c8936576b4df38c281c1aa4df5e2d/users/00105db60ffc41bcbd3d70ec9b4429c1/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:00.890502 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:00.891358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:00.891358 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:00.917798 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] MFA Rules not processed for user `00105db60ffc41bcbd3d70ec9b4429c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:00.919442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:00.931010 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:00.985450 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1355/2737] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:00.996504 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:00.999415 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:01.040361 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f619e30f070457f8f85738785898964', 'request_id': 'req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:01.047047 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f619e30f070457f8f85738785898964', 'request_id': 'req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:01.061241 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1f2c0-3817-4016-94bc-dca4b147b87d None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.109890 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.110530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.110813 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.111206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.131917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f619e30f070457f8f85738785898964', 'request_id': 'req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:01.144722 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] The token cache is being invalidate because project 3f619e30f070457f8f85738785898964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:01.145021 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:01.145724 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c99a7b90-fbad-4a09-aa40-d8fee5e721a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:01.147327 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1383/2738] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:00 2022] DELETE /identity/v3/projects/3f619e30f070457f8f85738785898964 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:01.153779 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.154649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.154649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.171733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.172768 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:01.173116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c29e1-65d1-4875-8f76-0afa68f6a959 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.174191 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:01.182538 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1356/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:37:01.245534 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1384/2740] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:01.255120 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.255491 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.255847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.273972 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:01.275287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:01.285479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1357/2741] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:01.289879 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.301321 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c3b7c-9b60-44b1-aa97-98277fed6213 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.306222 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.306880 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.307170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.307497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.308215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:01.308527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.310215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.322536 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1385/2742] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:37:01.327123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.336408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56acdf-3399-4bd3-a707-8b9abf365eb0 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.341850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.342279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.342561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.342840 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.343527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.345455 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.363718 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:01.447614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1358/2743] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:01 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:01.452972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.463247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46648c01-ef04-4fe0-9683-8cb4df0a8131 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.522586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.523270 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.523564 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.523850 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.524543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.526636 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.601614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1386/2744] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:01 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:01.605665 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.615603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281bccd-2f39-4411-b57c-fdcde9ed167d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.621291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.622084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.622603 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.623184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.624627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.626537 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.636739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1359/2745] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:01.641550 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.651745 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135790d0-bb68-4416-bf61-7e63453149de admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.657186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.658230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.658230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.658464 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.659144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.660753 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82205e-f568-4e76-882f-b9a1164fc34d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.671044 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1387/2746] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:01.685265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:01.695499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30bb21-795a-4ef6-9413-dcae239269e1 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:01.700642 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:01.701623 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.701903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.702180 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] PATH_INFO: `/v3/projects/8954ba71cdb84051b3e4d34b6fc48a0b/users/216a705cba89479581544af5e2dfb59c/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.719862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8954ba71cdb84051b3e4d34b6fc48a0b, user_id=216a705cba89479581544af5e2dfb59c, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:01.722608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:01.868107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1360/2747] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:01 2022] PUT /identity/v3/projects/8954ba71cdb84051b3e4d34b6fc48a0b/users/216a705cba89479581544af5e2dfb59c/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:01.885319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:01.885646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:01.885927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:01.905796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:01.914306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] MFA Rules not processed for user `216a705cba89479581544af5e2dfb59c`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:01.915538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:01.988052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1388/2748] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:01.999106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:02.000915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:02.061762 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962c0ec-2079-4930-bdaf-b2dcb051a8a2 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:02.065668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7017c7303f94b6380b99cb0b09d7e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:02.066334 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:02.068089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:02.068089 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:02.076849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:02.078380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52f7f3-5f39-42a1-802a-be9e750f4231 service glance] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:02.087486 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1361/2749] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:05.409110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:05.410698 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:05.479345 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b42ed-05d1-4b30-b6e7-a1d1878a0342 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:05.484365 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:05.485239 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:05.485558 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:05.485839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] PATH_INFO: `/v3/users/00105db60ffc41bcbd3d70ec9b4429c1` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:05.487049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00105db60ffc41bcbd3d70ec9b4429c1)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:05.489011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:05.779906 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00105db60ffc41bcbd3d70ec9b4429c1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:05.780652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00105db60ffc41bcbd3d70ec9b4429c1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:05.844037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00105db60ffc41bcbd3d70ec9b4429c1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:05.844664 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00105db60ffc41bcbd3d70ec9b4429c1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:05.859278 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00105db60ffc41bcbd3d70ec9b4429c1'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:05.872694 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1389/2750] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:05 2022] DELETE /identity/v3/users/00105db60ffc41bcbd3d70ec9b4429c1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:05.877547 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:05.878053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:05.886594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:05.907869 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:05.909101 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:05.975392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1362/2751] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:05.986813 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:05.988651 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:06.011650 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:06.051704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4997b6-fbb0-49f3-a192-1af35ab96052 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:06.055600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:06.056258 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:06.056543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:06.056822 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:06.067167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:06.068647 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73cbf0-afe2-4cd2-83d2-18309dd48360 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:06.079633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1390/2752] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:06.580070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:06.581595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:06.595831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901ced5-a35d-473e-8e96-537c3278681f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:06.601318 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:06.602286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:06.602572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:06.602849 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] PATH_INFO: `/v3/projects/fc7c8936576b4df38c281c1aa4df5e2d` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:06.604052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7c8936576b4df38c281c1aa4df5e2d)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:06.607444 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:06.753995 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7c8936576b4df38c281c1aa4df5e2d', 'request_id': 'req-4b9083b9-c199-4769-afa6-90a70f49b681'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:06.823660 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7c8936576b4df38c281c1aa4df5e2d', 'request_id': 'req-4b9083b9-c199-4769-afa6-90a70f49b681'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:06.826231 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:06.830206 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7c8936576b4df38c281c1aa4df5e2d', 'request_id': 'req-4b9083b9-c199-4769-afa6-90a70f49b681'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:06.841715 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] The token cache is being invalidate because project fc7c8936576b4df38c281c1aa4df5e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:06.842028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:06.842670 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b9083b9-c199-4769-afa6-90a70f49b681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:06.844461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1363/2753] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:06 2022] DELETE /identity/v3/projects/fc7c8936576b4df38c281c1aa4df5e2d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:06.849304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:06.849852 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:06.849977 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:06.869826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:06.871305 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:06.945676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1391/2754] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:06.954831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:06.955556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:06.956040 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:06.975610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:06.976839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:06.985574 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1364/2755] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:06.989924 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.000544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa022b0-0693-4d93-9b22-d71552f29a11 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.005246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.005928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.006214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.006490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.007215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:07.007523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.009146 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.023297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1392/2756] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:37:07.027634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.036974 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84344-0574-4cb5-af84-30a6f91d6a8f None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.041615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.042279 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.042560 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.042835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.043523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.046003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.153704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1365/2757] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:07 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:07.157396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.167444 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db881a5e-04c3-49c5-98e0-023c007cd1d1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.227606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.228272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.228559 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.228836 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.229555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.232349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.419763 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1393/2758] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:07 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:07.422819 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.433387 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e72d2-db29-4171-adfd-f5e57f730372 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.438320 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.439100 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.439413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.439692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.440414 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.442222 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.454581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1366/2759] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:07.458644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.468962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171273e-6184-42f1-993b-822ec61415da admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.474100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.474765 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.475043 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.475319 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.476040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.477868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f94f9-6faf-434d-91f7-9d222bc1299b admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.489058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1394/2760] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:07.493304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.504980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c6e25-aa7a-460a-850e-db38dc083c05 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.509643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.510534 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.510822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.511105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] PATH_INFO: `/v3/projects/f8bb35ed1e9440dbb0fe8bb80feb3982/users/ac5541cb2c684bf1850140fa3d766955/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.526743 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bb35ed1e9440dbb0fe8bb80feb3982, user_id=ac5541cb2c684bf1850140fa3d766955, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.530071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.624418 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1367/2761] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:07 2022] PUT /identity/v3/projects/f8bb35ed1e9440dbb0fe8bb80feb3982/users/ac5541cb2c684bf1850140fa3d766955/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:07.635815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e1e0-92c8-4403-9257-26bdec712a69 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.636259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e1e0-92c8-4403-9257-26bdec712a69 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.636812 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e1e0-92c8-4403-9257-26bdec712a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.667669 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-3a45e1e0-92c8-4403-9257-26bdec712a69 None None] MFA Rules not processed for user `ac5541cb2c684bf1850140fa3d766955`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:07.668957 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-3a45e1e0-92c8-4403-9257-26bdec712a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:07.681881 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:07.741489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1395/2762] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:07.752994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:07.754786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:07.819795 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6c145-13b9-403c-b72b-e624f533a74f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:07.823508 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:07.824186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:07.824533 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:07.824797 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:07.834478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:07.836039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:07.846873 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1368/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:10.736565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:10.737114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:10.737114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:10.759727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] MFA Rules not processed for user `ac5541cb2c684bf1850140fa3d766955`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:10.759727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:10.763903 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1396/2764] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:12.314572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:12.316047 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:12.378481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a968-0f17-4a0d-83ae-42dde0ac79f8 service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:12.382176 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:12.382829 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:12.383112 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:12.383386 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:12.385156 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:12.450223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:12.468971 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:12.470475 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:12.480004 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1369/2765] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:17.315013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:17.316579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:17.399038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a53b2e-7b9f-4289-8c58-ba76a8998f02 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:17.404702 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:17.406077 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:17.406507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:17.406801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:17.408661 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:17.426738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:17.428375 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364569-8f31-4297-ac55-736156f7f635 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:17.439633 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1397/2766] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:21.079911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:21.082104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:21.196323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccae17-8083-42c5-b02e-82ce9730745d service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:21.201880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:21.202862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:21.203149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:21.203424 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] PATH_INFO: `/v3/users/71c826d0cd104ecb8a9c8fbd2741aeb2` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:21.205000 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c826d0cd104ecb8a9c8fbd2741aeb2)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:21.207499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:21.584807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c826d0cd104ecb8a9c8fbd2741aeb2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:21.598213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c826d0cd104ecb8a9c8fbd2741aeb2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:21.701157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c826d0cd104ecb8a9c8fbd2741aeb2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:21.703230 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c826d0cd104ecb8a9c8fbd2741aeb2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:21.707889 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c826d0cd104ecb8a9c8fbd2741aeb2'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:21.723412 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1370/2767] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:21 2022] DELETE /identity/v3/users/71c826d0cd104ecb8a9c8fbd2741aeb2 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:21.725498 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:21.726750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:21.727100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:21.746135 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:21.747488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:21.820535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1398/2768] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:21.834852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:21.836529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:21.839940 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:21.841082 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:21.892704 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:21.910612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620f679-40ec-4189-b2f1-92415db74201 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:21.914646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2c4b5-cafa-416a-baab-34082a5837b4 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:21.914769 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:21.914895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:21.915169 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:21.915453 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:21.917703 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:21.918085 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:21.918349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:21.918736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:21.919100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:21.938716 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:21.940335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:21.940462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:21.940557 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:21.952825 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1371/2769] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:21.953532 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1399/2770] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:22.609980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:22.612086 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:22.622862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:22.630823 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ab889-72de-4591-94da-bdc02bf2d85a service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:22.635888 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:22.636862 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:22.643402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:22.643402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] PATH_INFO: `/v3/projects/43338785ced645639bb4f339f9b52cad` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:22.643402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43338785ced645639bb4f339f9b52cad)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:22.643402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:22.794221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43338785ced645639bb4f339f9b52cad', 'request_id': 'req-0f4e1203-76e5-4226-808b-c56ef10ddad6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:22.858331 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43338785ced645639bb4f339f9b52cad', 'request_id': 'req-0f4e1203-76e5-4226-808b-c56ef10ddad6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:22.870576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43338785ced645639bb4f339f9b52cad', 'request_id': 'req-0f4e1203-76e5-4226-808b-c56ef10ddad6'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:22.889664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] The token cache is being invalidate because project 43338785ced645639bb4f339f9b52cad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:22.890200 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:22.891037 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:22.896368 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1372/2771] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:22 2022] DELETE /identity/v3/projects/43338785ced645639bb4f339f9b52cad => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:22.896752 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:22.908056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7085a-36e8-4175-b618-059f39e4f519 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:22.978311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:22.979111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:22.979390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:22.979677 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] PATH_INFO: `/v3/users/42db7dff61e3438db72c33b3c105dd48` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:22.981905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42db7dff61e3438db72c33b3c105dd48)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:22.984617 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:23.191764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42db7dff61e3438db72c33b3c105dd48'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:23.192426 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42db7dff61e3438db72c33b3c105dd48'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:23.317542 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42db7dff61e3438db72c33b3c105dd48'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:23.318335 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42db7dff61e3438db72c33b3c105dd48'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:23.334840 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-5d31f42a-c302-47af-86da-69478699007e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42db7dff61e3438db72c33b3c105dd48'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:23.349982 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1400/2772] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:22 2022] DELETE /identity/v3/users/42db7dff61e3438db72c33b3c105dd48 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:24.282615 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:24.284516 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:24.304704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e1203-76e5-4226-808b-c56ef10ddad6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:24.387563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:24.388893 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:24.389379 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:24.396894 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] PATH_INFO: `/v3/projects/7f51707e654d41b3aadc10635c4685b9` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:24.399595 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f51707e654d41b3aadc10635c4685b9)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:24.401786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:24.517530 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f51707e654d41b3aadc10635c4685b9', 'request_id': 'req-718b3344-16a2-4037-8c6e-e615bc9a6dd7'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:24.603981 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f51707e654d41b3aadc10635c4685b9', 'request_id': 'req-718b3344-16a2-4037-8c6e-e615bc9a6dd7'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:24.611896 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f51707e654d41b3aadc10635c4685b9', 'request_id': 'req-718b3344-16a2-4037-8c6e-e615bc9a6dd7'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:24.625405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] The token cache is being invalidate because project 7f51707e654d41b3aadc10635c4685b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:24.625802 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:24.626467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-718b3344-16a2-4037-8c6e-e615bc9a6dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:24.628190 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1373/2773] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:24 2022] DELETE /identity/v3/projects/7f51707e654d41b3aadc10635c4685b9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:24.633710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:24.634313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:24.634438 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:24.654453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:24.656330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:24.726537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1401/2774] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:24.735926 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930e56-316b-47b5-be90-515994111f21 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:24.736487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930e56-316b-47b5-be90-515994111f21 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:24.736835 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d930e56-316b-47b5-be90-515994111f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:24.758286 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-1d930e56-316b-47b5-be90-515994111f21 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:24.759542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-1d930e56-316b-47b5-be90-515994111f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:24.770467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1374/2775] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:24.772990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:24.777913 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:24.783991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a2256-99bd-4a9e-af77-edbe9682e929 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:24.788841 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:24.789473 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:24.792453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:24.792453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:24.792453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:24.792453 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:24.793092 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:24.812580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1402/2776] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:37:24.817158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d930e56-316b-47b5-be90-515994111f21 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:24.827478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d930e56-316b-47b5-be90-515994111f21 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:24.832356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:24.833026 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:24.833306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:24.833692 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:24.834543 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:24.836668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:24.847104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:24.957939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1375/2777] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:24 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:37:24.962657 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:24.973938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38138545-2508-44dd-b9bc-61a1d66354ea admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.062136 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.064600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.064600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.064600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.065132 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.067231 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.236525 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1403/2778] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:24 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:37:25.237767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.250095 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b341e-edef-4b42-9ef6-2071037ee303 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.255005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.255714 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.256005 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.256293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.257020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.258697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.271102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1376/2779] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 16:37:25.275105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.286634 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463f23-b841-465e-afb8-80646d72e1f6 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.291652 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.292316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.292605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.292887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.293658 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.298339 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e4ec0-79b2-4005-95af-7c5edf9bc4ad admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.305828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1404/2780] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:25.310063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.321016 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468130f8-40a6-48ef-8673-9867c823a1b9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.325710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.326920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.326920 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.327986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] PATH_INFO: `/v3/projects/19171267b84c4a2784608724502e9f1c/users/3f3e6ed73dd84c5db84659d81bf58583/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.365405 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19171267b84c4a2784608724502e9f1c, user_id=3f3e6ed73dd84c5db84659d81bf58583, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.378438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.442435 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1377/2781] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:25 2022] PUT /identity/v3/projects/19171267b84c4a2784608724502e9f1c/users/3f3e6ed73dd84c5db84659d81bf58583/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:25.453493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.453493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.453829 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.487947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] MFA Rules not processed for user `3f3e6ed73dd84c5db84659d81bf58583`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:25.489492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:25.562723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1405/2782] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:25.564179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.628076 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf192ca-6095-4834-bf28-b6d7ee4ef41f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.632826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.633584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.633871 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.634149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.634831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.636834 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.738694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1378/2783] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:25 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:25.742739 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.756299 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36186c8-1a31-4cac-8672-d79031e05f95 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:25.844038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:25.851468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:25.855291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:25.855291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:25.855291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:25.855291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:25.856141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:25.981911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1406/2784] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:25 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:37:25.983160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:25.994521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d055e10-76a7-4bdd-9a0e-47be0754ec85 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.001864 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.004727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.004727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.004727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.004727 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.006102 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.019882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1379/2785] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:26.024543 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:26.035236 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8f9b-cf6c-4813-8e88-72c1f01ed7e3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.040285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.041169 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.041490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.041917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] PATH_INFO: `/v3/projects/70c229f6812c45e7903a61429646e087/users/d653b32863324b06b67b3cd0045395f0/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.057555 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c229f6812c45e7903a61429646e087, user_id=d653b32863324b06b67b3cd0045395f0, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.060265 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.147527 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1407/2786] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:26 2022] PUT /identity/v3/projects/70c229f6812c45e7903a61429646e087/users/d653b32863324b06b67b3cd0045395f0/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:26.158925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:26.163123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ed0aa-dcb5-48a8-adfe-8654e3901e68 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.226038 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.226801 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.227154 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.227499 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.228288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.229998 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbd0bf-6066-4496-b50d-cdf671914178 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.240392 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1380/2787] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:26.244971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:26.256127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec4a8-11d3-463b-91c6-d606d013f4c1 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.260804 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.261721 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.262005 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.262311 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] PATH_INFO: `/v3/projects/70c229f6812c45e7903a61429646e087/users/d653b32863324b06b67b3cd0045395f0/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.264968 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c229f6812c45e7903a61429646e087, user_id=d653b32863324b06b67b3cd0045395f0, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.267848 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.282984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1408/2788] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:26 2022] PUT /identity/v3/projects/70c229f6812c45e7903a61429646e087/users/d653b32863324b06b67b3cd0045395f0/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:26.293046 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.293440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.293733 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.323157 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] MFA Rules not processed for user `d653b32863324b06b67b3cd0045395f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:26.324439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:26.338397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:26.424915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1381/2789] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:26.511764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:26.513497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:26.580272 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f693f0-48bb-4503-918e-307b2382b399 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.584680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.585381 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.585551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.585831 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.595963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.597455 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e5c77-7d7f-46c6-9e64-7d40ed52c3e5 service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.607297 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1409/2790] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:26.638429 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:26.714703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe75eed-fcf8-4447-8b61-aafb067d8df7 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:26.714703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:26.714703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:26.714703 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:26.715478 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] PATH_INFO: `/v3/users/3f3e6ed73dd84c5db84659d81bf58583` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:26.715720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3e6ed73dd84c5db84659d81bf58583)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:26.717781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:26.965775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3e6ed73dd84c5db84659d81bf58583'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:26.979196 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3e6ed73dd84c5db84659d81bf58583'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:27.052810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3e6ed73dd84c5db84659d81bf58583'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:27.053785 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3e6ed73dd84c5db84659d81bf58583'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:27.054761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3e6ed73dd84c5db84659d81bf58583'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:27.070787 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1382/2791] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:26 2022] DELETE /identity/v3/users/3f3e6ed73dd84c5db84659d81bf58583 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:27.075025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:27.075537 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:27.075654 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:27.095746 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:27.096976 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:27.178807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1410/2792] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:27.185756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:27.187479 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:27.237039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:27.264467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588401-1912-4d35-98fe-dbf5bba4472f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:27.268831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:27.269519 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:27.269811 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:27.270087 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:27.281030 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:27.282643 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:27.293323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1383/2793] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:27.931561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:27.933601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:27.975710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd01bb-ca0b-4250-a54d-99cf502f3f88 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:27.982115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:27.983188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:27.984276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:27.984276 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] PATH_INFO: `/v3/projects/19171267b84c4a2784608724502e9f1c` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:27.985936 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19171267b84c4a2784608724502e9f1c)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:27.988680 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:28.147986 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:28.149757 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19171267b84c4a2784608724502e9f1c', 'request_id': 'req-33b3595a-1924-43c4-a49e-a2315ae8ff58'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.164990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19171267b84c4a2784608724502e9f1c', 'request_id': 'req-33b3595a-1924-43c4-a49e-a2315ae8ff58'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.258281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19171267b84c4a2784608724502e9f1c', 'request_id': 'req-33b3595a-1924-43c4-a49e-a2315ae8ff58'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.272758 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] The token cache is being invalidate because project 19171267b84c4a2784608724502e9f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:28.273217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.277376 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.280726 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1411/2794] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:27 2022] DELETE /identity/v3/projects/19171267b84c4a2784608724502e9f1c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:28.283148 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:28.332518 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db200b-e300-40b9-879c-8939af021d33 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:28.446417 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:28.447497 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:28.447989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:28.448476 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] PATH_INFO: `/v3/users/d653b32863324b06b67b3cd0045395f0` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:28.449963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d653b32863324b06b67b3cd0045395f0)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:28.453211 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:28.646172 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd653b32863324b06b67b3cd0045395f0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.667601 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd653b32863324b06b67b3cd0045395f0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.740786 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd653b32863324b06b67b3cd0045395f0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.745244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd653b32863324b06b67b3cd0045395f0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.745244 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-9ec9679e-2303-4768-8d7e-1f674a1b6d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd653b32863324b06b67b3cd0045395f0'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:28.762579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1384/2795] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:28 2022] DELETE /identity/v3/users/d653b32863324b06b67b3cd0045395f0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:29.537454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:29.538959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:29.553944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3595a-1924-43c4-a49e-a2315ae8ff58 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:29.638185 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:29.643141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:29.643141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:29.643141 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] PATH_INFO: `/v3/projects/70c229f6812c45e7903a61429646e087` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:29.644377 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c229f6812c45e7903a61429646e087)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:29.646586 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:29.804217 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c229f6812c45e7903a61429646e087', 'request_id': 'req-6f488acd-9c53-4a58-a6f8-6138cf0edadf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:29.812356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c229f6812c45e7903a61429646e087', 'request_id': 'req-6f488acd-9c53-4a58-a6f8-6138cf0edadf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:29.907109 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c229f6812c45e7903a61429646e087', 'request_id': 'req-6f488acd-9c53-4a58-a6f8-6138cf0edadf'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:29.930787 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] The token cache is being invalidate because project 70c229f6812c45e7903a61429646e087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:29.931091 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:29.934019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-6f488acd-9c53-4a58-a6f8-6138cf0edadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:29.938854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1412/2796] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:29 2022] DELETE /identity/v3/projects/70c229f6812c45e7903a61429646e087 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:29.945944 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:29.946521 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:29.946617 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:29.974260 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:29.975498 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:30.032470 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:30.050204 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1385/2797] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:30.059917 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.060213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.060497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.089778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:30.091226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:30.116054 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.121240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1413/2798] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:30.127898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91a737-c761-4411-a59b-40b40427fabe None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.133376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.136694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.136694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.136694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.136694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:30.136694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.138609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.151659 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1386/2799] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:37:30.156401 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.166038 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772047a-39eb-48f7-b04b-f9df4d68f24b None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.173054 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.173916 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.174320 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.174801 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.175462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.177562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.283796 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1414/2800] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:30 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:30.287909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.304261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a4f98-3c6f-44c7-a361-51e2c2658e37 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.388174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.388837 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.389126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.389667 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.390456 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.392724 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.541070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1387/2801] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:30 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:30.545631 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.557202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba734b9c-32ac-4285-9ac9-44e55e7f93fc admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.562395 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35164404-be19-4476-8ad9-519590398122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.563069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35164404-be19-4476-8ad9-519590398122 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.568863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35164404-be19-4476-8ad9-519590398122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.568863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35164404-be19-4476-8ad9-519590398122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.568863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35164404-be19-4476-8ad9-519590398122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.568863 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35164404-be19-4476-8ad9-519590398122 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.581971 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1415/2802] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:30.585599 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.601883 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60459e25-833d-43dc-84d5-b195d8ed8b4f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.605929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.606854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.607321 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.607750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.609291 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.611247 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b66fe1-9156-402e-a8ac-38fed7cc3a9c admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.653581 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1388/2803] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:30.661059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35164404-be19-4476-8ad9-519590398122 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:30.674727 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35164404-be19-4476-8ad9-519590398122 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:30.681816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:30.684587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.685202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.685890 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] PATH_INFO: `/v3/projects/c5c4acf47b96459f837ed605bfe07854/users/a1452f3a90844d56ac244b1fd10e706a/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.710261 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c4acf47b96459f837ed605bfe07854, user_id=a1452f3a90844d56ac244b1fd10e706a, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:30.713436 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:30.822980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1416/2804] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:30 2022] PUT /identity/v3/projects/c5c4acf47b96459f837ed605bfe07854/users/a1452f3a90844d56ac244b1fd10e706a/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:30.833903 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:30.834199 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:30.834480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:30.863809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] MFA Rules not processed for user `a1452f3a90844d56ac244b1fd10e706a`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:30.865302 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.012838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1389/2805] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:31.024682 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:31.026523 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.033078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:31.128649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb1ada-b71f-4c52-9afe-8e93d38a3d88 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:31.131984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:31.132643 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.132933 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.133215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.142949 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:31.146246 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:31.153104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:31.154628 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.159905 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1417/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:31.183611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:31.240961 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:31.262922 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854f2f0-4db0-47c9-93b3-25a599e874ec service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:31.268714 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:31.269883 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.270285 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.270624 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] PATH_INFO: `/v3/users/a1452f3a90844d56ac244b1fd10e706a` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.274880 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6736762-0a89-4b08-9e2c-b36c1dd80224 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:31.275068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1452f3a90844d56ac244b1fd10e706a)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:31.275068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:31.277053 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:31.277781 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.278075 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.278362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.280949 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.304616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:31.304616 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:31.316885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1390/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:37:31.328788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:31.330398 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.349119 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1215a8f-2110-4469-bc24-0d80bcdcf019 service placement] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:31.360839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:31.360839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.360839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.360839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] PATH_INFO: `/v3/users/672b81726d1445518823f6e7c8887978` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.361370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] RBAC: Authorizing `identity:delete_user(user_id=672b81726d1445518823f6e7c8887978)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:31.361370 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:31.484816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1452f3a90844d56ac244b1fd10e706a'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.485843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1452f3a90844d56ac244b1fd10e706a'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.612355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1452f3a90844d56ac244b1fd10e706a'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.612832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1452f3a90844d56ac244b1fd10e706a'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.627846 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-077f653c-ace4-4391-9bf3-e031acfff557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1452f3a90844d56ac244b1fd10e706a'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.628066 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672b81726d1445518823f6e7c8887978'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.639774 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672b81726d1445518823f6e7c8887978'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.640783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1418/2808] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:31 2022] DELETE /identity/v3/users/a1452f3a90844d56ac244b1fd10e706a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:31.645142 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.645461 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.645741 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.662972 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:31.664110 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.723390 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1419/2809] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:31.731955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672b81726d1445518823f6e7c8887978'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.732686 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672b81726d1445518823f6e7c8887978'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.733989 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672b81726d1445518823f6e7c8887978'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:31.737315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:31.737315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.747287 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1391/2810] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:31 2022] DELETE /identity/v3/users/672b81726d1445518823f6e7c8887978 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:31.803447 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0548b7-892c-4508-a5c4-f15e836b5629 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:31.854049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:31.854735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:31.855019 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:31.855296 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:31.857117 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:31.877484 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:31.929851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:31.931471 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:31.942608 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1420/2811] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:32.088739 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:32.093477 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:32.104442 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bae900-3741-4206-84b5-cbc59b8547df admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:32.111166 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:32.112138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:32.112591 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:32.112693 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] PATH_INFO: `/v3/projects/e57270a9020844e9baf044630fb81bcd` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:32.114056 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57270a9020844e9baf044630fb81bcd)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:32.122540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:32.552020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57270a9020844e9baf044630fb81bcd', 'request_id': 'req-2869324d-7925-4b56-8cac-322b31bc8d1d'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.681772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:32.683332 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:32.696226 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8685-cce4-4743-930f-c34d37daac52 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:32.715461 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57270a9020844e9baf044630fb81bcd', 'request_id': 'req-2869324d-7925-4b56-8cac-322b31bc8d1d'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.727848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57270a9020844e9baf044630fb81bcd', 'request_id': 'req-2869324d-7925-4b56-8cac-322b31bc8d1d'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.729898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=100 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:32.740311 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] The token cache is being invalidate because project e57270a9020844e9baf044630fb81bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:32.740657 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.741420 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.742994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1392/2812] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:32 2022] DELETE /identity/v3/projects/e57270a9020844e9baf044630fb81bcd => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:32.746872 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:32.748725 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:32.760859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:32.761777 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:32.763007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:32.763298 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] PATH_INFO: `/v3/projects/c5c4acf47b96459f837ed605bfe07854` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:32.764533 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c4acf47b96459f837ed605bfe07854)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:32.767480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:32.783003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:32.817640 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869324d-7925-4b56-8cac-322b31bc8d1d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:32.822438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:32.823345 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:32.823646 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:32.823905 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] PATH_INFO: `/v3/users/dd7aaaa3c51c43c7ab27d122363c37aa` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:32.825096 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7aaaa3c51c43c7ab27d122363c37aa)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:32.827097 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:32.903108 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c4acf47b96459f837ed605bfe07854', 'request_id': 'req-27e9a66a-2fb8-418c-bd3e-28ee488e485d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:32.908354 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c4acf47b96459f837ed605bfe07854', 'request_id': 'req-27e9a66a-2fb8-418c-bd3e-28ee488e485d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.047143 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c4acf47b96459f837ed605bfe07854', 'request_id': 'req-27e9a66a-2fb8-418c-bd3e-28ee488e485d'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.059312 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] The token cache is being invalidate because project c5c4acf47b96459f837ed605bfe07854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:33.059616 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.063103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.063103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1421/2813] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:32 2022] DELETE /identity/v3/projects/c5c4acf47b96459f837ed605bfe07854 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:33.154672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7aaaa3c51c43c7ab27d122363c37aa'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.170550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7aaaa3c51c43c7ab27d122363c37aa'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.281146 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7aaaa3c51c43c7ab27d122363c37aa'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.281874 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7aaaa3c51c43c7ab27d122363c37aa'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.282492 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-5caa8d89-4b41-4928-88da-756171f6360d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7aaaa3c51c43c7ab27d122363c37aa'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.307729 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1393/2814] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:32 2022] DELETE /identity/v3/users/dd7aaaa3c51c43c7ab27d122363c37aa => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:33.606621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:33.608080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:33.681380 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9a66a-2fb8-418c-bd3e-28ee488e485d admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:33.739719 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:33.740579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:33.740865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:33.741139 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] PATH_INFO: `/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:33.742341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40eb92d00fc34b498c8c0a44f27e2e15)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:33.744245 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:33.754788 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:33.885330 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40eb92d00fc34b498c8c0a44f27e2e15', 'request_id': 'req-c85cce89-5e23-4490-a47c-b5cef07e072c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.892535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40eb92d00fc34b498c8c0a44f27e2e15', 'request_id': 'req-c85cce89-5e23-4490-a47c-b5cef07e072c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:33.997755 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40eb92d00fc34b498c8c0a44f27e2e15', 'request_id': 'req-c85cce89-5e23-4490-a47c-b5cef07e072c'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:34.012177 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] The token cache is being invalidate because project 40eb92d00fc34b498c8c0a44f27e2e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:34.012482 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:34.013202 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:34.015348 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1422/2815] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:33 2022] DELETE /identity/v3/projects/40eb92d00fc34b498c8c0a44f27e2e15 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:37.237999 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff382-f5c8-466c-9b30-a6dcc325c6ba None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:37.238481 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff382-f5c8-466c-9b30-a6dcc325c6ba None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:37.238596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ff382-f5c8-466c-9b30-a6dcc325c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:37.257425 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c13ff382-f5c8-466c-9b30-a6dcc325c6ba None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:37.258709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c13ff382-f5c8-466c-9b30-a6dcc325c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:37.315856 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1394/2816] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:37:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:37.464614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:37.466131 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:37.539386 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cce89-5e23-4490-a47c-b5cef07e072c admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:37.544411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:37.545058 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:37.545367 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:37.545649 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:37.555747 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:37.557363 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:37.567178 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1423/2817] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:37:38.509503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03fce-91aa-4af8-88ac-94b8b453ed3f None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:38.509983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03fce-91aa-4af8-88ac-94b8b453ed3f None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:38.510103 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03fce-91aa-4af8-88ac-94b8b453ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:38.528243 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-c4d03fce-91aa-4af8-88ac-94b8b453ed3f None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:38.529487 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-c4d03fce-91aa-4af8-88ac-94b8b453ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:38.537382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1395/2818] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:37:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:38.550538 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:38.561188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6fae0-86ed-43fe-a7fe-bec6cc2e0189 service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:38.566325 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:38.566962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:38.567252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:38.567535 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:38.575598 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:38.577181 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:38.585379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1424/2819] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 16:37:39.261485 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f32ab-8480-457c-9a69-938b559d1551 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:39.261784 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f32ab-8480-457c-9a69-938b559d1551 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:39.262067 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f32ab-8480-457c-9a69-938b559d1551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:39.281093 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-cc9f32ab-8480-457c-9a69-938b559d1551 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:39.282526 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-cc9f32ab-8480-457c-9a69-938b559d1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:39.289775 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1396/2820] 10.222.0.42 () {58 vars in 1009 bytes} [Fri May 27 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:39.301477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:39.312015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54af38-6db1-4e99-b45f-3537bb6423a0 service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:39.318504 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:39.323151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:39.323151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:39.323151 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:39.328133 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:39.330488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:39.338970 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1425/2821] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:37:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:37:40.049408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:40.050018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:40.050207 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:40.069709 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] MFA Rules not processed for user `ebdc88eed0ec4c19af2d34413c4f73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:40.070942 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:40.078441 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1397/2822] 10.222.0.42 () {58 vars in 967 bytes} [Fri May 27 16:37:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:40.090708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:40.102808 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b226-c3d0-4471-be46-a85b42a7a2a4 service swift] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:40.107984 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c95e2d016a84b5a804f4d18816a81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:40.108708 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:40.108946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:40.109308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:40.122415 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:40.130612 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de21d5-b2a5-4fdd-8b6a-eded9f05b3db service swift] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:40.145433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1426/2823] 10.222.0.42 () {60 vars in 1384 bytes} [Fri May 27 16:37:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 16:37:40.733557 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:40.734963 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:40.788365 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:40.821163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0710-6abe-48f7-a4ed-b634603fb1e6 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:40.827440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:40.828307 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:40.829003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:40.829003 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] PATH_INFO: `/v3/users/216a705cba89479581544af5e2dfb59c` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:40.830780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] RBAC: Authorizing `identity:delete_user(user_id=216a705cba89479581544af5e2dfb59c)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:40.833240 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:41.036980 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216a705cba89479581544af5e2dfb59c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.053099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216a705cba89479581544af5e2dfb59c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.132305 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216a705cba89479581544af5e2dfb59c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.132939 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216a705cba89479581544af5e2dfb59c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.133611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216a705cba89479581544af5e2dfb59c'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.147766 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1398/2824] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:40 2022] DELETE /identity/v3/users/216a705cba89479581544af5e2dfb59c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:41.152528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:41.153011 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:41.153106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:41.171928 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:41.173134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:41.227798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:41.243723 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1427/2825] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:41.254007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:41.255402 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:41.315039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554cce5-0d0a-49b7-b04f-bdbe833a186d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:41.318915 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:41.319610 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:41.319885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:41.320185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:41.336357 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:41.337965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434d656-990e-4c3a-afdd-51c7d3639f0a service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:41.350282 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1399/2826] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:41.802620 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:41.804279 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:41.818807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ed508-0ec5-4e32-a337-9ed6e886d353 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:41.829304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:41.830871 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:41.831356 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:41.831795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] PATH_INFO: `/v3/projects/8954ba71cdb84051b3e4d34b6fc48a0b` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:41.833439 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8954ba71cdb84051b3e4d34b6fc48a0b)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:41.838193 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:41.993056 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8954ba71cdb84051b3e4d34b6fc48a0b', 'request_id': 'req-ca9255b0-2b82-473e-9e1d-bd577d2547bd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:41.998681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8954ba71cdb84051b3e4d34b6fc48a0b', 'request_id': 'req-ca9255b0-2b82-473e-9e1d-bd577d2547bd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:42.137518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8954ba71cdb84051b3e4d34b6fc48a0b', 'request_id': 'req-ca9255b0-2b82-473e-9e1d-bd577d2547bd'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:42.149646 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] The token cache is being invalidate because project 8954ba71cdb84051b3e4d34b6fc48a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:42.149960 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:42.150593 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-ca9255b0-2b82-473e-9e1d-bd577d2547bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:42.152134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1428/2827] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:41 2022] DELETE /identity/v3/projects/8954ba71cdb84051b3e4d34b6fc48a0b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:42.157078 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.157566 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.157754 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.175983 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:42.177183 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:42.228452 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:42.240690 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1400/2828] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:42.249088 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.249579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.249693 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.268707 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:42.269938 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:42.277815 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1429/2829] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:42.284361 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.293986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ddf8a-3aa4-42c9-adf7-39a30e81bbc3 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.299570 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.299790 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.300077 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.300351 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] PATH_INFO: `/v3/domains` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.301068 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:42.301407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.303011 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.314467 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1401/2830] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 16:37:42.319076 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.328997 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1224e-41c2-484c-8a9e-7877209073d6 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.334243 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.334896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.335318 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.335613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.336396 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.338350 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.448590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1430/2831] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:42 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:42.452576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.463115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7364da-89b6-44c7-b8d8-c5dfd6e6faad admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.520909 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.521638 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.521927 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.522210 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.522929 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.524855 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.619020 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1402/2832] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:42 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:42.623606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.634247 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a6aa-85d1-41d0-829c-b56460a569a9 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.638847 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.639544 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.639832 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.640114 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.640839 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.643274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.654223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1431/2833] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:42.659679 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.669664 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec00ee0-354c-4bb2-a75e-b7d43f0ac6c6 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.674272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.674994 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.675283 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.675587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.676319 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.677952 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e98585-a9e9-4c04-950e-e0ba79c507b6 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.687863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1403/2834] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:42.692506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.702785 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac48a0-858f-45b5-b000-4b351d55aba3 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.708302 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.709174 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.709493 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.709772 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] PATH_INFO: `/v3/projects/1d237a862db54f8fa6349c169d09d902/users/87a88415b21b4507ac744ee3f6f5d82e/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.726611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d237a862db54f8fa6349c169d09d902, user_id=87a88415b21b4507ac744ee3f6f5d82e, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.729393 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.800932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1432/2835] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:42 2022] PUT /identity/v3/projects/1d237a862db54f8fa6349c169d09d902/users/87a88415b21b4507ac744ee3f6f5d82e/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:42.811454 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.811750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.812035 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.838569 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] MFA Rules not processed for user `87a88415b21b4507ac744ee3f6f5d82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:42.839832 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:42.894811 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.896520 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:42.911967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1404/2836] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:42.924750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:42.926147 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:42.970798 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79852ae8-4949-4e10-9f43-e6e9c6480fe2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.975904 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.977101 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.977452 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.977736 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] PATH_INFO: `/v3/users/ac5541cb2c684bf1850140fa3d766955` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.978926 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5541cb2c684bf1850140fa3d766955)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:42.980901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:42.986592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e44bc5-55f4-4628-925a-00473b3c771e None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:42.990245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:42.990898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:42.991184 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:42.991464 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:42.992251 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:42.996803 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:43.000636 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:43.002756 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd18cc-b918-4615-adf3-45537fc5bafd service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:43.012039 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1405/2837] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 16:37:43.206519 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5541cb2c684bf1850140fa3d766955'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.207215 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5541cb2c684bf1850140fa3d766955'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.271959 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5541cb2c684bf1850140fa3d766955'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.272587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5541cb2c684bf1850140fa3d766955'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.285402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5541cb2c684bf1850140fa3d766955'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.297946 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1433/2838] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:42 2022] DELETE /identity/v3/users/ac5541cb2c684bf1850140fa3d766955 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:43.303127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:43.303624 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:43.303841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:43.322277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:43.323496 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:43.384045 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1406/2839] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:43.393914 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:43.395446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:43.454288 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475215-ff45-4db4-b18d-21992b3deb50 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:43.458454 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:43.459170 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:43.459456 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:43.459735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:43.469237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:43.471374 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b376-e884-4d6d-b6f7-82b115faea06 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:43.480854 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1434/2840] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:43.711857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:43.713380 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:43.737614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35994d61-eea3-4f67-8dbd-9c993542c0c9 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:43.742383 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:43.743229 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:43.743523 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:43.743807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] PATH_INFO: `/v3/projects/f8bb35ed1e9440dbb0fe8bb80feb3982` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:43.745015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bb35ed1e9440dbb0fe8bb80feb3982)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:43.753559 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:43.858897 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bb35ed1e9440dbb0fe8bb80feb3982', 'request_id': 'req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:43.995123 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bb35ed1e9440dbb0fe8bb80feb3982', 'request_id': 'req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:44.002815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bb35ed1e9440dbb0fe8bb80feb3982', 'request_id': 'req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:44.015107 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] The token cache is being invalidate because project f8bb35ed1e9440dbb0fe8bb80feb3982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:44.015407 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:44.016071 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-61a8f98e-bee9-4db6-9ca4-5b06cae969cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:44.018857 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1407/2841] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:43 2022] DELETE /identity/v3/projects/f8bb35ed1e9440dbb0fe8bb80feb3982 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:44.023756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.024059 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.024391 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.043153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:44.044313 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:44.108882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1435/2842] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:44.117771 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.118070 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.118353 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.138063 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:44.140185 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:44.147397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1408/2843] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:44.151552 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.161775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e11114-875e-4bf7-8ac3-43cf57c63a25 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.166433 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.167080 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.167366 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.167644 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.168397 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:37:44.168706 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.170344 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.182349 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1436/2844] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:37:44.186698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.196763 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991d2bd-d717-4201-a06b-49836e068071 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.201622 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.202277 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.202562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.202843 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.203527 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.205542 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.290698 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1409/2845] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:37:44 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:37:44.295154 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.306006 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b746c-9924-4918-99ff-c4a354fb0734 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.320346 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:44.377886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.378700 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.378878 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.379153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.379857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.381820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.488199 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1437/2846] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:37:44 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 16:37:44.492594 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.504104 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffbe3f-9286-4031-8384-9014b6c4b2b4 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.508951 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.509654 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.510189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.510561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.511376 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.513069 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.525099 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1410/2847] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:44.529268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.540026 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c612444-38c4-4386-a74c-e2d9d98f80a8 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.544944 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.545713 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.545945 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.546252 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.547001 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.549260 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d308f09-6465-4f95-b927-21d9f7c3b790 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.559590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1438/2848] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:37:44.563791 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.573793 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed05c7-d2e6-4c04-bd94-155342b3ea5b admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.578245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.579117 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.579395 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.579800 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] PATH_INFO: `/v3/projects/f4b5ddb45f284ece84c7a94eb95a11c1/users/c946427352794703a848399e29f524b6/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.594548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b5ddb45f284ece84c7a94eb95a11c1, user_id=c946427352794703a848399e29f524b6, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.598160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.717732 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1411/2849] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:37:44 2022] PUT /identity/v3/projects/f4b5ddb45f284ece84c7a94eb95a11c1/users/c946427352794703a848399e29f524b6/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 16:37:44.723308 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906cc-e89a-4c2d-8e9c-7c65c679caae None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.723611 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906cc-e89a-4c2d-8e9c-7c65c679caae None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.723962 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d906cc-e89a-4c2d-8e9c-7c65c679caae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.742462 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-91d906cc-e89a-4c2d-8e9c-7c65c679caae None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:44.743686 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-91d906cc-e89a-4c2d-8e9c-7c65c679caae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:44.806733 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1439/2850] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:44.820650 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:44.822480 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:44.838694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:44.889681 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323da621-f6de-4f61-b8c4-92d8543afec9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:44.893621 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:44.894272 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:44.894562 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:44.894847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:44.903965 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:44.905440 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:44.915235 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1412/2851] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:46.514888 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977324cc-9713-4779-b394-d2981ec14b64 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:46.515416 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977324cc-9713-4779-b394-d2981ec14b64 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:46.515568 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977324cc-9713-4779-b394-d2981ec14b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:46.543809 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-977324cc-9713-4779-b394-d2981ec14b64 None None] MFA Rules not processed for user `c946427352794703a848399e29f524b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:46.545122 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-977324cc-9713-4779-b394-d2981ec14b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:46.610340 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1440/2852] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:46.623248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:46.624973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:46.682808 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02549133-5170-4d62-ba73-b3596b1b3ba6 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:46.686495 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:46.687144 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:46.687465 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:46.687742 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:46.697116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:46.698596 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:46.707824 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1413/2853] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:49.617002 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196be5-eec9-4ba4-b34c-fc0e7bd460df None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:49.617843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196be5-eec9-4ba4-b34c-fc0e7bd460df None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:49.617843 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196be5-eec9-4ba4-b34c-fc0e7bd460df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:49.636499 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-cb196be5-eec9-4ba4-b34c-fc0e7bd460df None None] MFA Rules not processed for user `c946427352794703a848399e29f524b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:49.637748 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-cb196be5-eec9-4ba4-b34c-fc0e7bd460df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:49.645492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1441/2854] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 16:37:52.041882 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:52.043780 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:52.060158 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1ebb9-c65b-4927-93e3-ef597b35eafa service cinder] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:52.065514 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:52.066381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:52.066688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:52.066997 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] PATH_INFO: `/v3/users/87a88415b21b4507ac744ee3f6f5d82e` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:52.068214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a88415b21b4507ac744ee3f6f5d82e)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:52.070175 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:52.237252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a88415b21b4507ac744ee3f6f5d82e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:52.249720 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a88415b21b4507ac744ee3f6f5d82e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:52.328548 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a88415b21b4507ac744ee3f6f5d82e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:52.330736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a88415b21b4507ac744ee3f6f5d82e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:52.330736 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a88415b21b4507ac744ee3f6f5d82e'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:52.343576 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1414/2855] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:37:52 2022] DELETE /identity/v3/users/87a88415b21b4507ac744ee3f6f5d82e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:37:52.348040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:52.350932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:52.350932 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:52.367039 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:37:52.368281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:52.383626 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:52.436429 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1442/2856] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:37:52.444836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:52.447015 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:52.485389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:37:52.511445 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98b651-4624-4e30-a2ec-5d71836b019d admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:52.515363 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:52.516019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:52.516300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:52.516579 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:52.527174 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:52.528614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:52.538208 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1415/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:37:53.036497 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:37:53.038119 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:37:53.050902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e15a47-768b-4f97-994b-c665c85b6a19 None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:37:53.055756 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:37:53.057514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:37:53.057514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:37:53.057514 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] PATH_INFO: `/v3/projects/1d237a862db54f8fa6349c169d09d902` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:37:53.059259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d237a862db54f8fa6349c169d09d902)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:37:53.061420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:37:53.232079 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d237a862db54f8fa6349c169d09d902', 'request_id': 'req-1467b881-e299-4d62-a7f9-4228772ae33f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:53.237995 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d237a862db54f8fa6349c169d09d902', 'request_id': 'req-1467b881-e299-4d62-a7f9-4228772ae33f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:53.317186 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d237a862db54f8fa6349c169d09d902', 'request_id': 'req-1467b881-e299-4d62-a7f9-4228772ae33f'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:53.330216 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] The token cache is being invalidate because project 1d237a862db54f8fa6349c169d09d902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80566) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:37:53.330512 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:53.331188 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:37:53.332775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1443/2858] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:37:53 2022] DELETE /identity/v3/projects/1d237a862db54f8fa6349c169d09d902 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:38:02.863633 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=12 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:38:03.326179 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:38:50.731753 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:38:50.733153 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:38:50.789809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65a613-762a-45ac-bb87-06d4a31b4566 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:38:50.838362 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:38:50.839013 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:38:50.839300 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:38:50.839584 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:38:50.841573 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:38:50.911292 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:38:50.912821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ecbee6-d8bb-4e57-9b3c-12d473ca0aea service cinder] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:38:50.923820 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1416/2859] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:38:53.909095 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:38:53.910673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:38:53.968274 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467b881-e299-4d62-a7f9-4228772ae33f admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:38:53.971896 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:38:53.972587 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:38:53.972886 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:38:53.973156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:38:53.974963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:38:53.989738 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:38:53.992379 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:38:54.003728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1444/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:38:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:39:00.918895 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=74 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:39:03.998505 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=26 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:39:07.913684 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:39:07.914841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:39:07.914841 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:39:07.932161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] MFA Rules not processed for user `c946427352794703a848399e29f524b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:39:07.933458 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:07.990838 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1417/2861] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:39:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:39:08.226127 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:39:08.234947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580acc-748a-4394-9493-1e29e9bfcc2c service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:39:08.238529 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:39:08.239190 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:39:08.239475 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:39:08.239750 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:39:08.241518 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:08.251560 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:39:08.252990 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:39:08.261487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1445/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:39:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:39:08.676164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:39:08.677605 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:08.759936 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86b949-661e-4df2-94fb-e0f6b50dd202 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:39:08.764885 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:39:08.765561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:39:08.765850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:39:08.766132 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:39:08.767900 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:08.780413 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:39:08.782059 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:39:08.793018 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1418/2863] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:39:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:39:11.712267 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:39:11.713930 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:11.724565 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ead2-59c8-4711-8311-05b9fd95c734 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:39:11.728825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:39:11.729942 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:39:11.730227 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:39:11.730503 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:39:11.732293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:11.743656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:39:11.745105 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:39:11.754315 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1446/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:39:12.568694 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:39:12.570323 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:12.586430 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314bbad-9e20-49bc-b95f-bf4b50b3cdb9 service placement] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:39:12.592846 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:39:12.593540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:39:12.593826 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:39:12.594105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:39:12.595911 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:39:12.609697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:39:12.611360 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:39:12.628588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1419/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:39:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:39:21.750063 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=12 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:39:22.618541 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=63 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:16.698528 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:16.699778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:16.833540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b119-7207-4548-a773-66e6dbd89757 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:16.838445 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:16.839316 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:16.839605 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:16.839887 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] PATH_INFO: `/v3/users/c946427352794703a848399e29f524b6` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:16.841118 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c946427352794703a848399e29f524b6)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:16.843103 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:17.059980 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c946427352794703a848399e29f524b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.060642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c946427352794703a848399e29f524b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.137085 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c946427352794703a848399e29f524b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.138040 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c946427352794703a848399e29f524b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.150898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-7478b145-6e68-4e1c-b60e-f06c0f8ec55e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c946427352794703a848399e29f524b6'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.163734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1447/2866] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:40:16 2022] DELETE /identity/v3/users/c946427352794703a848399e29f524b6 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:40:17.401711 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:17.441265 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310cf108-6427-4873-8664-af91baa5e06d service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:17.498149 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:17.499007 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:17.499293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:17.499571 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] PATH_INFO: `/v3/projects/f4b5ddb45f284ece84c7a94eb95a11c1` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:17.500831 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b5ddb45f284ece84c7a94eb95a11c1)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:17.502818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:17.611604 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b5ddb45f284ece84c7a94eb95a11c1', 'request_id': 'req-20aeadf1-aafc-4c6e-b730-26e06e7e4368'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.709327 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b5ddb45f284ece84c7a94eb95a11c1', 'request_id': 'req-20aeadf1-aafc-4c6e-b730-26e06e7e4368'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.715550 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b5ddb45f284ece84c7a94eb95a11c1', 'request_id': 'req-20aeadf1-aafc-4c6e-b730-26e06e7e4368'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.727561 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] The token cache is being invalidate because project f4b5ddb45f284ece84c7a94eb95a11c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:40:17.727860 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.730116 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-20aeadf1-aafc-4c6e-b730-26e06e7e4368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:40:17.730250 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1420/2867] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:40:17 2022] DELETE /identity/v3/projects/f4b5ddb45f284ece84c7a94eb95a11c1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:40:17.735206 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:17.735734 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:17.735849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:17.764580 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:17.785897 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:17.787551 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:17.848876 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1448/2868] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:17.858607 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:17.858899 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:17.859181 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:17.880036 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:17.881288 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:17.889160 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1421/2869] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:17.893784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:17.904277 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f5492-5b07-4c74-a8e5-b6d6661390be None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:17.908898 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:17.909573 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:17.909861 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:17.910147 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:17.911078 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80566) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 16:40:17.911572 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:17.913187 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:17.924857 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1449/2870] 10.222.0.42 () {58 vars in 1152 bytes} [Fri May 27 16:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 16:40:17.929572 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:17.939285 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66ca54-3400-4185-be4c-3f11d709dda1 None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:17.944608 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:17.945214 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:17.945529 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:17.945810 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:17.946503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:17.948446 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.041041 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1422/2871] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:40:17 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:40:18.045407 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:18.056506 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0cf2-3d88-412b-839b-f558dc9bb547 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:18.112480 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:18.113134 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.113509 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.114673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.115676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:18.117614 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.194775 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1450/2872] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:40:18 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:40:18.199163 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:18.210248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e6c66-9a83-4836-840b-411c14e6adc5 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:18.214850 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:18.215515 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.215818 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.216098 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.216815 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:18.218587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.229588 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1423/2873] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:40:18.233764 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:18.244479 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e320b-8b91-408f-8993-612769e78585 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:18.249300 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:18.249978 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.250282 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.250579 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.251306 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:18.252895 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2cc341-75ed-48e2-b8b5-610c04c33f77 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.263590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1451/2874] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:40:18.267853 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:18.279777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990b18-fb75-4008-89b6-cdbf51bcb12a admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:18.284494 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:18.285408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.286356 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.286750 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] PATH_INFO: `/v3/projects/a7987c5244084c328b132a11290832ba/users/b1c1a7980121436885dfc548322658fe/roles/87496757e0464ade96d2c6d9ef7ca8ec` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.301671 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7987c5244084c328b132a11290832ba, user_id=b1c1a7980121436885dfc548322658fe, role_id=87496757e0464ade96d2c6d9ef7ca8ec)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:18.304388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.401209 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1424/2875] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:40:18 2022] PUT /identity/v3/projects/a7987c5244084c328b132a11290832ba/users/b1c1a7980121436885dfc548322658fe/roles/87496757e0464ade96d2c6d9ef7ca8ec => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:40:18.406710 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100703e-2b4d-4435-b610-d7a4d6a789d4 None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.407010 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100703e-2b4d-4435-b610-d7a4d6a789d4 None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.407295 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100703e-2b4d-4435-b610-d7a4d6a789d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.426902 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-2100703e-2b4d-4435-b610-d7a4d6a789d4 None None] MFA Rules not processed for user `e387d1cbdd3947bcb9659065cbae32c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:18.428197 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-2100703e-2b4d-4435-b610-d7a4d6a789d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:18.489441 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1452/2876] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:18.499343 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:18.501025 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:18.503865 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:18.603564 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac4e7b-f419-4758-b682-04c442220047 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:18.607248 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:18.607898 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:18.608189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:18.608474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:18.618289 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:18.619735 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:18.630704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1425/2877] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:40:23.784988 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:23.786115 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:23.786218 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:23.792402 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:23.822607 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.auth.core [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] MFA Rules not processed for user `b1c1a7980121436885dfc548322658fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80566) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:23.823740 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:23.899761 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1453/2878] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:23.904687 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:23.933459 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4eba7-b16c-47b7-91a9-81758e75d4c9 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:23.939655 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:23.940347 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:23.940814 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:23.941106 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:23.946108 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:23.948755 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.060462 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1426/2879] 10.222.0.42 () {60 vars in 1152 bytes} [Fri May 27 16:40:23 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:40:24.064718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:24.076991 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea08e9-7bd5-43c7-8bc0-a27eb03c268a None None] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:24.139489 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:24.140167 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] REQUEST_METHOD: `POST` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:24.140673 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:24.140805 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] PATH_INFO: `/v3/users` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:24.141600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:24.143472 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.241144 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1454/2880] 10.222.0.42 () {60 vars in 1143 bytes} [Fri May 27 16:40:24 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 16:40:24.246001 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:24.257788 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2e390-cfe9-46b2-9b82-784edcbf3084 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:24.262474 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:24.263164 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:24.263432 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:24.263712 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:24.264439 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:24.266105 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.278528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1427/2881] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:40:24.282621 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:24.293632 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeb683-db92-415c-827f-d71df51ca0ce admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:24.298676 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:24.299663 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:24.310507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:24.310507 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] PATH_INFO: `/v3/projects/af2941d22bda4f96ae25b40dc475daf5/users/5de6c0755dc741f7945c546a28817efc/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:24.321783 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2941d22bda4f96ae25b40dc475daf5, user_id=5de6c0755dc741f7945c546a28817efc, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:24.324656 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.428106 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1455/2882] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:40:24 2022] PUT /identity/v3/projects/af2941d22bda4f96ae25b40dc475daf5/users/5de6c0755dc741f7945c546a28817efc/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:40:24.433495 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:24.450922 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9641c-0244-4c48-856e-549f786e264f admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:24.527662 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:24.528433 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:24.528783 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:24.529152 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:24.529986 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:24.531821 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459d074-1566-434d-8521-3070144a0274 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.543839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1428/2883] 10.222.0.42 () {58 vars in 1121 bytes} [Fri May 27 16:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 16:40:24.547628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:24.558823 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ac49d-9e99-46a2-8207-09bc03df276a admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:24.564562 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:24.565488 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] REQUEST_METHOD: `PUT` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:24.565784 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:24.566064 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] PATH_INFO: `/v3/projects/af2941d22bda4f96ae25b40dc475daf5/users/5de6c0755dc741f7945c546a28817efc/roles/c60a3fdf3263401aa67021ec3e540337` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:24.568628 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2941d22bda4f96ae25b40dc475daf5, user_id=5de6c0755dc741f7945c546a28817efc, role_id=c60a3fdf3263401aa67021ec3e540337)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:24.571406 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:24.584901 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1456/2884] 10.222.0.42 () {60 vars in 1482 bytes} [Fri May 27 16:40:24 2022] PUT /identity/v3/projects/af2941d22bda4f96ae25b40dc475daf5/users/5de6c0755dc741f7945c546a28817efc/roles/c60a3fdf3263401aa67021ec3e540337 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:40:30.829219 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce8cb-efb3-4cae-9aa2-8fec9b5f34a1 None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:30.829799 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce8cb-efb3-4cae-9aa2-8fec9b5f34a1 None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:30.829891 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fce8cb-efb3-4cae-9aa2-8fec9b5f34a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:30.862765 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-b7fce8cb-efb3-4cae-9aa2-8fec9b5f34a1 None None] MFA Rules not processed for user `5de6c0755dc741f7945c546a28817efc`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:30.865245 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-b7fce8cb-efb3-4cae-9aa2-8fec9b5f34a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:30.893563 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:30.950836 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1429/2885] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:30.966911 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:30.968728 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:31.022915 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:31.052237 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08c487-8f41-4c50-a81b-3a3af48d85db admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:31.059372 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:31.065240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:31.065240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:31.065240 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:31.128041 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:31.131069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:31.142253 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1457/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:40:40.970919 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=19 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:41.137799 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=36 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:40:51.954822 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:51.955337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:51.959221 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:51.975609 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] MFA Rules not processed for user `b1c1a7980121436885dfc548322658fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:40:51.976002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:51.983680 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1430/2887] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:40:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:40:52.296037 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:40:52.351466 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1a977-c5f9-4b47-b940-701c8a1aba68 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:40:52.355166 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:40:52.355807 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:40:52.356153 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:40:52.356368 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:40:52.358214 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:40:52.367941 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:40:52.369394 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:40:52.378615 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1458/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:41:01.968704 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:41:02.374584 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=26 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:41:09.112549 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:41:09.114428 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:41:09.138577 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0a4e0-ded2-4d1d-b807-8b48bf5eae1c None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:41:09.144126 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:41:09.144710 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:41:09.145012 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:41:09.145270 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:41:09.147091 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:41:09.159266 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:41:09.160848 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:41:09.171580 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1431/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:41:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:41:11.987927 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:41:11.996891 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93df062-3abc-46d1-9de8-fc259bba8724 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:41:12.001111 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:41:12.001849 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:41:12.002138 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:41:12.002420 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:41:12.011446 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:41:12.012882 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:41:12.022967 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1459/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:41:13.946934 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:41:13.959767 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8046-06ed-4044-ba48-c4394ab55e3f service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:41:13.965460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:41:13.966049 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:41:13.967019 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:41:13.967460 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:41:13.969252 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:41:13.981925 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:41:13.983967 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c692970-fbc4-4d17-aa01-d7a27f82a2ea service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:41:13.995337 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1432/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:41:22.018792 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:41:23.990391 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=15 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:42:54.225865 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:42:54.227268 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:42:54.319183 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1240-5b30-4fc3-ad42-c1dad6f5f48d service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:42:54.324370 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:42:54.325007 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:42:54.325289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:42:54.325590 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:42:54.327326 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:42:54.339754 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:42:54.341606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:42:54.351516 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1460/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:42:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:43:04.347259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=41 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:43:15.290150 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e7e9-80ea-459a-a6e6-8448ce85fbdb None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:43:15.290649 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e7e9-80ea-459a-a6e6-8448ce85fbdb None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:43:15.290761 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e7e9-80ea-459a-a6e6-8448ce85fbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:43:15.308304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-8d37e7e9-80ea-459a-a6e6-8448ce85fbdb None None] MFA Rules not processed for user `b1c1a7980121436885dfc548322658fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:43:15.312381 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-8d37e7e9-80ea-459a-a6e6-8448ce85fbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:43:15.316806 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1433/2893] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:43:25.303434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=3 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:44:51.844692 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:44:51.846235 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:44:51.858900 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926f95c-eee8-4d91-84e0-42bde1c64fc4 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:44:51.862613 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:44:51.863248 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:44:51.864983 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:44:51.865281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:44:51.867212 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:44:51.876958 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:44:51.878440 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:44:51.887294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1461/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:01.883601 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:45:13.143138 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] REQUEST_METHOD: `POST` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:13.143812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:13.143812 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:13.162179 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.auth.core [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] MFA Rules not processed for user `b1c1a7980121436885dfc548322658fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 16:45:13.163382 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:13.170170 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1434/2895] 10.222.0.42 () {58 vars in 957 bytes} [Fri May 27 16:45:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 16:45:13.207281 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:13.208816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:13.287210 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009ff40-13fa-4ad7-b864-387cf4ecf440 service neutron] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:13.290953 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2be41ec2de3948e6a015dfe35c47a0a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:13.291662 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:13.291947 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:13.292230 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:13.294031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:13.322049 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:13.323487 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:13.343540 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1462/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:13.650852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:13.660978 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94124a56-ed80-4cb7-982b-f263352a42ab None None] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:13.664488 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:13.665115 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:13.665503 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:13.665782 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:13.667506 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:13.677858 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:13.679281 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:13.687186 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1435/2897] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:14.394062 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:14.404389 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:14.406223 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d449c8-875a-4e7a-9bc1-1d4468c708ed service cinder] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:14.412522 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:14.413259 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:14.413592 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd5dfb-697c-4782-961a-7da9fc10da22 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:14.413718 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:14.414013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:14.415826 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:14.417177 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:14.419278 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:14.419565 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:14.419847 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:14.421688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:14.428492 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:14.430175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:14.436084 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:14.437551 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:14.441642 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1463/2898] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:14.450308 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1436/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:18.304081 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:18.319851 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d88403-758a-47b4-b7aa-53a80cc69796 service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:18.326490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:18.327152 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:18.327609 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:18.328068 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:18.330825 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:45:18.344293 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:18.346213 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:18.359954 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1464/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:24.444612 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=15 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:45:28.354160 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=56 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:45:30.648421 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:45:30.659992 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096b5a-177f-4b0d-803f-623e14cf0fc6 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:45:30.665672 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:45:30.666304 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:45:30.666587 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:45:30.666863 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:45:30.675717 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:45:30.677397 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:45:30.686438 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1437/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:45:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:45:40.682434 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=6 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:47:14.611468 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:14.628069 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73af32a-c411-426a-a823-d4a575da0317 service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:14.636477 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb545deab49645c5947c3eacd618dc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:14.637175 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:14.637457 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:14.637912 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:14.651013 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:14.652639 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:14.665189 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1465/2902] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:47:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:47:24.659989 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=6 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:47:30.352955 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:30.363697 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef6ab2-84fd-4b35-9a0d-2f21133f1d85 service nova] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:30.367553 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e62e836ab6e4f42814b3151bfe686d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:30.368201 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:30.368489 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:30.368777 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:30.431668 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:30.433109 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:30.442854 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1438/2903] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:47:34.078610 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:34.093681 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a02edf-5584-433d-94d0-6ab6349a542c service nova] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:34.101004 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:34.101735 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:34.102025 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:34.102304 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:34.104120 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:47:34.117600 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:34.119360 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:34.131599 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1466/2904] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:47:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:47:37.743611 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:37.759028 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8525d5a-469b-4ed5-859b-4816b40ddfe5 service neutron] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:37.766189 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:37.767493 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] REQUEST_METHOD: `GET` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:37.768062 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:37.768696 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:37.770406 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:47:37.784259 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:37.787002 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:37.799388 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1439/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:47:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:47:37.897561 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:37.909289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc56955-98fa-46ca-834a-988b97c3083f service placement] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:37.914963 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:37.915868 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:37.916155 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:37.916432 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] PATH_INFO: `/v3/users/b1c1a7980121436885dfc548322658fe` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:37.917778 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1c1a7980121436885dfc548322658fe)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:37.920289 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:38.088648 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1c1a7980121436885dfc548322658fe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.089098 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1c1a7980121436885dfc548322658fe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.147052 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1c1a7980121436885dfc548322658fe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.148065 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1c1a7980121436885dfc548322658fe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.164490 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1c1a7980121436885dfc548322658fe'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.178596 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1467/2906] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:47:37 2022] DELETE /identity/v3/users/b1c1a7980121436885dfc548322658fe => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:47:38.453127 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:38.469768 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8660a3-5da3-42ac-9bb7-08df1998414c service placement] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:38.533816 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:38.534707 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:38.535024 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:38.535306 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] PATH_INFO: `/v3/projects/a7987c5244084c328b132a11290832ba` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:38.536634 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7987c5244084c328b132a11290832ba)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:38.538627 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:38.659730 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7987c5244084c328b132a11290832ba', 'request_id': 'req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.725852 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7987c5244084c328b132a11290832ba', 'request_id': 'req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.737973 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7987c5244084c328b132a11290832ba', 'request_id': 'req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.758556 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] The token cache is being invalidate because project a7987c5244084c328b132a11290832ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:47:38.759213 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.760034 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:38.762688 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1440/2907] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:47:38 2022] DELETE /identity/v3/projects/a7987c5244084c328b132a11290832ba => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:47:38.769031 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:38.770554 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:47:38.785328 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f2a9a-b588-4b8b-9532-6caceabe0ab2 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:38.861820 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:38.863310 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:38.863816 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:38.864322 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] PATH_INFO: `/v3/users/5de6c0755dc741f7945c546a28817efc` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:38.866291 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de6c0755dc741f7945c546a28817efc)` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:38.868254 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:39.189015 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de6c0755dc741f7945c546a28817efc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.190294 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de6c0755dc741f7945c546a28817efc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.288828 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de6c0755dc741f7945c546a28817efc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.289481 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de6c0755dc741f7945c546a28817efc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.295221 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:47:39.302919 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.notifications [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de6c0755dc741f7945c546a28817efc'} {{(pid=80566) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.315192 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1468/2908] 10.222.0.42 () {58 vars in 1223 bytes} [Fri May 27 16:47:38 2022] DELETE /identity/v3/users/5de6c0755dc741f7945c546a28817efc => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:47:39.567912 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Authenticating user token {{(pid=80565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:47:39.569400 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.fernet_utils [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:47:39.582809 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82501ef2-8e80-4cf9-86ed-ce4b9784dca9 admin admin] Validating token access rules against request {{(pid=80565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:47:39.591223 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:47:39.651408 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e387d1cbdd3947bcb9659065cbae32c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b89eb764079a4814b557772bebc9a51d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:47:39.652293 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:47:39.652702 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:47:39.652839 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] PATH_INFO: `/v3/projects/af2941d22bda4f96ae25b40dc475daf5` {{(pid=80565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:47:39.654073 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2941d22bda4f96ae25b40dc475daf5)` {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:47:39.656052 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] RBAC: Authorization granted {{(pid=80565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:47:39.775341 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2941d22bda4f96ae25b40dc475daf5', 'request_id': 'req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.840540 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2941d22bda4f96ae25b40dc475daf5', 'request_id': 'req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.847422 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2941d22bda4f96ae25b40dc475daf5', 'request_id': 'req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343'} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.859261 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] The token cache is being invalidate because project af2941d22bda4f96ae25b40dc475daf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 16:47:39.859807 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.860528 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG keystone.notifications [None req-f6a9dab8-74e2-4eb8-8d4c-0f7408896343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 16:47:39.862614 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: [pid: 80565|app: 0|req: 1441/2909] 10.222.0.42 () {58 vars in 1232 bytes} [Fri May 27 16:47:39 2022] DELETE /identity/v3/projects/af2941d22bda4f96ae25b40dc475daf5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 16:47:49.503100 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:47:49.856161 dv-cinder-840493-2-iscsi devstack@keystone.service[80565]: DEBUG dbcounter [-] [80565] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 16:48:00.125859 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Authenticating user token {{(pid=80566) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 16:48:00.127273 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:48:00.196795 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a182e-3349-4db5-a524-0553227fb298 admin admin] Validating token access rules against request {{(pid=80566) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 16:48:00.257411 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1f644a909746fe92bd0dee4e9ce635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bb133900b13417cb2458c5ccd8e62d5', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80566) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 16:48:00.258055 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] REQUEST_METHOD: `GET` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 16:48:00.258341 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] SCRIPT_NAME: `/identity` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 16:48:00.258618 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80566) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 16:48:00.260413 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.fernet_utils [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80566) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 16:48:00.271606 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 16:48:00.273156 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c127e9-119f-4021-a581-ecc5f5bf9089 service placement] RBAC: Authorization granted {{(pid=80566) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 16:48:00.283355 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: [pid: 80566|app: 0|req: 1469/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Fri May 27 16:48:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 16:48:10.278810 dv-cinder-840493-2-iscsi devstack@keystone.service[80566]: DEBUG dbcounter [-] [80566] Writing DB stats keystone:SELECT=58 {{(pid=80566) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}